Chapter 21. Services for PCI Drivers

The IRIX 6.5 kernel provides a uniform interface for managing a PCI device. The functions in this interface are covered in this chapter under the following headings:

IRIX 6.5 PCI Drivers

This section discusses changes made to PCI driver support for IRIX 6.5 and refers to the appropriate sections of the manual for more information.

About PCI Drivers

A PCI device driver is a kernel-level device driver that has the general structure described in Chapter 7, “Structure of a Kernel-Level Driver”. It uses the driver/kernel interface described in Chapter 8, “Device Driver/Kernel Interface”. A PCI driver can be loadable or it can be linked with the kernel. In general it is configured into IRIX as described in Chapter 9, “Building and Installing a Driver”.

PCI hardware configuration is more dynamic than the configuration of the VME or EISA buses. With other types of bus, the administrator describes the device configuration using VECTOR statements and the configuration is static. IRIX support for the PCI bus is designed to allow support for dynamic reconfiguration. A PCI driver can be designed to allow devices to be attached and detached at any time.

The general sequence of operations of a PCI driver is as follows:

  1. In the pfxinit() entry point, the driver prepares any global variables.

  2. In the pfxreg() entry point, the driver calls a kernel function to register itself as a PCI driver, specifying the kind of device it supports.

  3. When the kernel discovers a device of this type, it calls the pfxattach() entry point of the driver.

  4. In the normal upper-half entry points such as pfxopen(), pfxread(), and pfxstrategy(), the driver operates the device and transfers data.

  5. If the kernel learns that the device is being detached, the kernel calls the driver's pfxdetach() entry point. The driver undoes the work done in by pfxattach().

A PCI driver uses a number of PCI-related kernel functions that are all declared in the header file sys/PCI/pciio.h.

About Registration

Registration is a step that lets the kernel know how to associate a device to a driver.

A PCI device identifies itself on the bus by its vendor ID and device ID numbers. The kernel discovers the complement of devices by probing the bus. When it finds a device, the kernel needs to associate it with a driver. With other types of bus, the association between a device and a driver is entered by the system administrator in a static configuration file. For PCI devices, the kernel looks through a list of drivers that have registered as supporting PCI devices of particular types.

Your driver registers by calling the pciio_driver_register() function (see reference page pciio(d3) ).This call specifies the PCI vendor ID and device ID numbers as they appear in the PCI configuration space of any device that this driver can support. The third argument is the driver's prefix string as configured in its descriptive file (see “Describing the Driver in /var/sysgen/master.d” in Chapter 9). The kernel uses this string to find the addresses of the driver's pfxattach() and pfxdetach() entry points.

Example 21-1 shows a hypothetical example of driver registration. This fragmentary example also shows how a driver can register multiple times to handle multiple combinations of vendor ID and device ID.

Example 21-1. Driver Registration

int hypo_reg()
{
   ret = pciio_driver_register(HYPO_VENID,HYPO_DEVID1,"hypo_",0);
   if (!ret)
   {
      cmn_err(CE_WARN,"error %d registering devid %d",ret,HYPO_DEVID1);
      return ret;
   }
   ret = pciio_driver_register(HYPO_VENID,HYPO_DEVID2,"hypo_",0);
   if (!ret)...
}

In a loadable driver, you must call pciio_driver_register() from the pfxreg() entry point. In a nonloadable driver, you can make the call from pfxinit() if you prefer, but the driver might not then work if someone later tries to make it loadable.

Wherever you call the function, be aware that, if there is an available device of the specified type, pfxattach() can be called immediately, before the pci_driver_register() function returns. In a multiprocessor, pfxattach() can be called concurrently with the return of pci_driver_register() and following code.

About Attaching a Device

The duties and actions of the pfxattach() entry point are discussed in detail in “Entry Point attach()” in Chapter 7 and under “Hardware Graph Management” in Chapter 8. In summary, at this time the driver

  • Creates hwgraph vertexes to represent the device

  • Allocates and initializes a data structure to hold per-device information

  • Allocates PIO maps and (optionally) DMA maps to use in addressing the device

  • If necessary, registers an interrupt handler

  • If necessary, registers an error handler

  • Initializes the device itself

The allocation and use of PIO and DMA maps, and the registration of an interrupt handler, are covered in detail in following topics.

The argument to pfxattach() is a hwgraph vertex handle that represents the “connection point” of the device—usually the bus slot. The driver builds more vertexes connected to this one to represent the logical device. However, the handle of the connection point is needed in several kernel functions, and it should be saved as part of the device information.

The return code from pfxattach() is tested by the kernel. The driver can reject an attachment. When your driver cannot allocate memory, or fails due to another problem, it should:

  • Use cmn_err() to document the problem (see “Using cmn_err” in Chapter 10)

  • Release any objects such as PIO and DMA maps that were created

  • Release any space allocated to the device such as a device information structure

  • Return an informative return code

The pfxdetach() entry point can only be called if the pfxattach() entry point returns success (0).

More than one driver can register to support the same vendor ID and device ID. The order in which drivers are called to attach a device is not defined. When the first-called driver fails to complete the attachment, the kernel continues on to test the next, until all have refused or one accepts.

About Unloading

When a loadable PCI driver is called at its pfxunload() entry point, indicating that the kernel would like to unload it, the driver must take pains not to leave any dangling pointers (as discussed under “Entry Point unload()” in Chapter 7). A driver should not unload when it has any registered interrupt or error handlers.

A driver does not have to unregister itself as a PCI driver before unloading. Nor does it have to detach any devices it has attached. However, if any devices are open or memory mapped, the driver should not unload.

If the driver has been autoregistered (see “Registration” in Chapter 9), stub functions are placed in the switch tables for the attach and open functions. When the kernel discovers a new device and wants this driver to attach it, or when a process attempts to open a device for which this driver created the vertex, the kernel reloads the driver. 

Using PIO Maps

You use a PIO map to establish a mapping between a kernel virtual address and some portion of PCI bus memory space, configuration space, or I/O space so that the CPU can load and store into the PCI bus. Depending on the machine architecture, the mapping may be a simple, linear translation, or it may require the kernel to program hardware registers in one or more bus adapters. The software interface is the same in all cases.

You cannot program a PCI device without at least one PIO map and you might allocate several. Typically you store the handles of the allocated maps in the device information structure; and you store the address of the device information structure in turn in the hwgraph vertex for the device.

In summary, a PIO map is used as follows:

  1. Allocate it with pciio_piomap_alloc().

  2. Activate the map and extract a translated address using pciio_piomap_addr(). Use the translated address to fetch or store.

  3. Deactivate the map using pciio_piomap_done(), when the map will be kept but will not be used for some time.

  4. Release the map with pciio_piomap_free().

PIO Mapping Functions

The functions that are used to create and apply PIO maps are summarized in Table 21-1. For syntax details see the reference page pciio_pio(d3).

Table 21-1. Functions for PIO Maps for the PCI Bus

Function

Header Files

Purpose and Operation

pciio_piomap_alloc()

ddi.h, pciio.h

Create a PIO map object, specifying the bus address space, base offset, and length it needs to cover.

pciio_piomap_addr()

ddi.h, pciio.h

Get a kernel virtual address from a PIO map for a specific offset and length.

pciio_piomap_done()

ddi.h, pciio.h

Make a PIO map inactive until it is next needed (may release hardware resources associated to the map).

pciio_piomap_free()

ddi.h, pciio.h

Release a PIO map object.

pciio_piotrans_addr()

ddi.h, pciio.h

Request immediate translation of a bus address to a kernel virtual address without use of a PIO map. Returns NULL unless this system supports fixed PIO addressing.

pciio_pio_addr()

ddi.h, pciio.h

Attempt immediate translation, but allocate a PIO map if necessary.

pciio_piospace_alloc()

ddi.h, pciio.h

Reserve a segment of PCI bus memory or I/O space.

pciio_piospace_free()

ddi.h, pciio.h

Release a segment of PCI bus memory or I/O space.


Allocating PIO Maps

You create a PIO map using pciio_piomap_alloc(). Its arguments are as follows (see also reference page pciio_pio(d3) ):

vhdl 

The connection-point vertex_hdl_t received by the pfxattach() routine. This handle identifies the device to the kernel by its bus and slot positions.

dev_desc 

Device descriptor structure (see text following).

space 

Constant specifying the space to map (see Table 21-2

 and text).

addr 

Offset within the selected space (typically 0).

size 

Span of the total area in space over which this map might be applied.

max 

Maximum size of the area that will be mapped at any one time.

flags 

Optional usage flags (no-sleep flag, PCIIO_BYTE_STREAM, and so on)

Example 21-2 shows a function that allocates a PIO map. The address space is passed as an argument, as is the size of the space to map. The function assumes the map should start at offset 0 in the selected space.

Example 21-2. Allocation of PCI PIO Map

#include <sys/PCI/pciio.h>
pciio_piomap_t makeMap(vertex_hdl_t connpt, int space, size_t size)
{
   return pciio_piomap_alloc(
         connpt,      /* connection point vertex handle */
         device_desc_default_get(convpt), /* device_desc_t */
         space,       /* space, typically _WIN(n) */
         0,           /* starting offset */
         size,size,   /* size to map */
         0);          /* sleeping is OK */
}


Preparing a device_desc_t

The device descriptor structure type device_desc_t is declared in iobus.h, which is included by pciio.h (see also reference page device_desc(d4x) ). In this release there is little that the device driver needs to know about this structure and its contents. The simplest way to get a device descriptor that can be handed to pciio_piomap_alloc() is to call device_desc_default_get() passing the same connection-point vertex handle, as follows:

ret = pciio_piomap_alloc(convh,device_desc_default_get(convh),...)

Selecting the Address Space

The space argument of pciio_piomap_alloc() specifies the address space to which this PIO map can apply. The possible choices are summarized in Table 21-2.

Table 21-2. PIO Map Address Space Constants

Constant Name

Meaning

PCIIO_PIOMAP_WIN(n)

The memory space defined by the BAR word n in configuration space. This is the most common type of PIO map base. (See note below.)

PCIIO_SPACE_WIN(n)

The memory space defined by the BAR word n in configuration space. This is the most common type of PIO map base.

PCIIO_SPACE_CFG

The Configuration address space. Direct PIO access to configuration space is supported only in IRIX 6.4—see “Changes In Configuration Interface”

.

PCIIO_SPACE_IO

Map to an absolute addr in the PCI bus I/O address space.

PCIIO_SPACE_MEM

Map to an absolute addr in the PCI bus memory address space. PCI memory space is usually preallocated by the IRIX kernel; use this only when space has been allocated with pciio_piospace_alloc(); see “Allocating PIO Bus Space”

.



Note: PCIIO_MAP_* flags are being replaced by PCIIO_SPACE_* flags. Both are still supported but we recommend you change to using PCIIO_SPACE_* flags if you have not already done so.

The space selection PCIIO_PIOMAP_WIN(n) means that this map is to be based on Base Address Register (BAR) n, from 0 through 5, in the PCI configuration space. If this selects a BAR that decodes I/O space, the map is for I/O space. Typically this selects a BAR that decodes memory space. When the space is defined by a 64-bit base address register, use the lower number that indexes the word that contains the configuration bits.


Note: The PCI infrastructure verifies that a segment of size max, starting at addr, can be mapped in the specified space, based on the device configuration. If this is not possible, the map is not allocated and NULL is returned.


Sizing the Space

The max argument sets a limit on the total span of addresses, from lowest to highest, for which this map can ever be used. When the map is always used for the same area, size and max are the same. When the map can be used for smaller segments within a larger area, size is the limit of any single segment and max the size of the total extent. The size to be mapped at any one time is specified when you apply pciio_piomap_addr() to the allocated map.

Specifying the No-Sleep Flag

The pciio_piomap_alloc() function may need to allocate memory. Normally it does so with a function that can sleep if memory is temporarily unavailable. If it is important that the function never sleep, pass PCIIO_NOSLEEP in the flags argument. When you do this, you must check for a NULL return, indicating that memory was not available.

Allocating PIO Bus Space

When the kernel locates a PCI device on the bus, it allocates the amount of PCI bus Memory and I/O address space that the device requests in its standard configuration registers. You get a PIO map into this preallocated space by allocating a map for space PCIIO_SPACE_WIN(n), specifying the configuration register for that space.

In some cases, a PCI device needs additional memory or I/O space based on device-specific configuration data that is not known to the kernel. You use pciio_piospace_alloc() to allocate additional ranges of memory or I/O space to be used by a device. It is up to your driver to program the device to use the allocated space.

When you need to perform PIO to allocated space that is not decoded by the standard BARs, you create a PIO map for space PCIIO_SPACE_MEM or PCIIO_SPACE_IO, and specify the exact base address that was allocated.

Performing PIO With a PIO Map

After a map has been allocated, it is inactive. The function pciio_piomap_addr() activates a map if it is not active, and uses the map to translate an offset within the mapped space to a kernel virtual address.

In some systems, “activating a map” can be a null operation. In other systems, an active PIO map may represent a commitment of limited hardware resources—for example, a mapping register in a bus adapter. The function arguments are as follows (see also reference page pciio_pio(d3) :

map 

The allocated map to use. The map specifies the address space.

addr 

The offset in the mapped space.

size 

The number of bytes to be mapped.

If any argument is invalid, or if the map cannot be activated, the returned address is 0. The returned address, when it is not 0, can be used to fetch and store from the PCI bus as if it were memory. An attempt to access beyond the specified size might cause a kernel panic or might simply return bad data.

Accessing Memory and I/O Space

PIO access to memory or I/O space follows the same pattern: extract a translated address using pciio_piomap_addr(), then use the address as a memory pointer. The function in Example 21-3 encapsulates the process of reading a word based on a map.

Example 21-3. Function to Read Using a Map

__uint_32_t mapRefer(pciio_piomap_t map, iopaddr_t offset)
{
   volatile __uint32_t *xaddr;   /* word in PCI space */
   xaddr = pciio_piomap_addr(map,offset,sizeof(*xaddr));
   if (xaddr)
      return *xaddr;
   cmn_err(CE_WARN,"Unable to map PCI PIO address");
   return 0xffffffff; /* imitate hardware fault */
}

Access to quantities smaller than 32 bits needs special handling. When you access a 16-bit or 8-bit value, the least-significant address bits must reflect the PCI byte-lane enable bits. What this means in practice is that the target address of a 16-bit value must be exclusive-ORed with 0x02, and the target address of an 8-bit value must be exclusive-ORed with 0x03. You can do this explicitly, by modifying the word address returned from pciio_piomap_addr(). Alternatively you can use the PIO address to base a structure, and in the structure you can invert the positions of bytes and halfwords within words, so that the sum of base and offset has the correct PIO address.

Deactivating an Address and Map

After you extract an address using pciio_piomap_addr(), the map is active, supporting the translated address over the span of bytes you specified. The address remains valid only as long as the map supports it.

The address becomes inactive when you call pciio_piomap_addr() for a different address or size based on the same map. If you attempt to use an address after the map has changed, a kernel panic can occur.

The map itself remains active until you call either pciio_piomap_done() or pciio_piomap_free(). In some systems, it costs nothing to keep a PIO map active. In other systems, an active PIO map may tie up global hardware resources. It is a good idea to call pciio_piomap_done() when the current address will not be used for some time. 

Using One-Step PIO Translation

Some systems also support a one-step translation function, pciio_piotrans_addr(). This function takes a combination of the arguments of pciio_piomap_alloc() and pciio_piomap_addr(), and returns a translated address. In effect, it combines creating a map, using the map, and freeing the map, into a single step (see reference page pciio_pio(d3) ).

This function can fail in systems that do not use hard-wired bus maps. If you use it, you must test the returned address. If it is 0, the one-step translation failed. The address is invalid, and you must create a PIO map instead.

The two-step process of allocating a map and then interrogating it is more general and works in all systems.

Accessing the Device Configuration

Typically a PCI driver needs to read the device configuration registers and possibly write to them. These are PIO operations, but the interface for performing them has varied between releases.

Changes In Configuration Interface

The hardware to generate PCI configuration cycles differs from one system to another. In all systems, access to configuration space is limited to 32-bit words on 32-bit boundaries. For these and other reasons, configuration access methods have varied from release to release.

  • In IRIX 6.3, configuration access was done by obtaining a PIO map address for configuration space and passing it to kernel functions pciio_config_get() and pciio_config_set(). This is because, in the O2 workstation, configuration cycles are not generated by normal PIO operations. The functions operate the special hardware.

  • In IRIX 6.4, you performed configuration access using PIO through a PIO map. This is because, in the hardware supported by IRIX 6.4 (Origin, Onyx2, and Octane), normal PIO access through a map can generate PCI configuration cycles, although only for 32-bit transfers.

  • As of this release, IRIX 6.5, you are required to use functions pciio_config_get() and pciio_config_set() for configuration access. This is because that release supports the O2 as well as other platforms. However, the interface to these functions is extended to support 8-byte registers, transfers of 1, 2, and 3 bytes, and access to nonstandard (device-defined) configuration registers.

In order to bring a degree of uniformity to this picture, the macros in Example 21-4 are presented.

Example 21-4. Configuration Access Macros

/* PCI Config Space Access Macros for source compatibility in drivers
** that use the same source for IRIX 6.3, IRIX 6.4, and IRIX 6.5
** Usage:
**    PCI_CFG_BASE(conn)
**    PCI_CFG_GET(conn,base,offset,type)
**    PCI_CFG_SET(conn,base,offset,type,value)
**
** Use caddr_t cfg_base = PCI_CFG_BASE(c) once during attach to get the
** PIO base address for the specific device needed by 6.3 and 6.4.
** Later, use PCI_CFG_GET() to read and PCI_CFG_SET() to write config registers.
**
** NOTE: IRIX 6.4 supports only 32-bit access. IRIX 6.3 determines the size of
** register (1-4 bytes) based on the offset and the standard layout of a Type 00
** PCI Configuration Space Header. If you specify a nonstandard size or offset,
** you get different results in different releases.
*/
#if IRIX6_3
#define PCI_CFG_BASE(c)        pciio_piotrans_addr(c,0,PCIIO_SPACE_CFG,0,256,0)
#define PCI_CFG_GET(c,b,o,t)   pciio_config_get(b,o)
#define PCI_CFG_SET(c,b,o,t,v) pciio_config_set(b,o,v)
#elif IRIX6_4
#define PCI_CFG_BASE(c)        pciio_piotrans_addr(c,0,PCIIO_SPACE_CFG,0,256,0)
#define PCI_CFG_GET(c,b,o,t)   ((*(t *)((char *)(b)+(o))))
#define PCI_CFG_SET(c,b,o,t,v) ((*(t *)((char *)(b)+(o))) = v)
#else /* IRIX 6.5 and onward */
#define PCI_CFG_BASE(c)  NULL
#define PCI_CFG_GET(c,b,o,t)   pciio_config_get(c,o,sizeof(t))
#define PCI_CFG_SET(c,b,o,t,v) pciio_config_set(c,o,sizeof(t),v)
#endif

The skeletal code in Example 21-5 illustrates access to 32-bit words in configuration space using the macros.

Example 21-5. Reading PCI Configuration Space

int hypo_attach(vertex_hdl_t connpt)
{
...
   pDevInfo->conn_vh = connpt;
...
   pDevInfo->cfg_base = PCI_CFG_BASE(connpt);
...
}
__uint32_t get_config_word(DevInfo *pDefInfo, int offset)
{
   return PCI_CFG_GET(pDevInfo->conn_vh,
                      pDevInfo->cfg_base,
                      offset, sizeof(__uint32_t));
}
void set_config_word(DevInfo *pDefInfo, int offset, __uint32_t val)
{
   PCI_CFG_SET(pDevInfo->conn_vh,
               pDevInfo->cfg_base,
               offset,sizeof(val),val);
}


Interrogating PIO Maps

The following functions can be used to interrogate a PIO map object (see pciio_get(d3) ):

Table 21-3. Functions for Interrogating PIO Maps

Function

Header Files

Purpose and Operation

pciio_pio_dev_get()

ddi.h, pciio.h

Return the connection point handle from a map.

pciio_pio_mapsz_get()

ddi.h, pciio.h

Return the maximum size of a map. (Note that this returns a ulong as of IRIX 6.5.)

pciio_pio_pciaddr_get()

ddi.h, pciio.h

Return the bus base address for a map.

pciio_pio_space_get()

ddi.h, pciio.h

Return the specified bus address space of a map.

pciio_pio_slot_get()

ddi.h, pciio.h

Return the bus slot number of the mapped device.

Most of these functions return values that were supplied to pciio_piomap_alloc(). However, pciio_pio_slot_get() provides a way to learn the bus slot number of a device. You can also obtain the slot number from a general query function; see “Interrogating a PCI Device”

PCI Drivers for the O2 (IP32) Platform

For IRIX 6.5 drivers on the O2 (IP32) platform, new function prototypes are provided for PIO access. These are defined in sys/PCI/pciio.h as follows:

extern uint8_t     pciio_pio_read8(volatile uint8_t *addr);
extern uint16_t    pciio_pio_read16(volatile uint16_t *addr);
extern uint32_t    pciio_pio_read32(volatile uint32_t *addr);
extern uint64_t    pciio_pio_read64(volatile uint64_t *addr);
extern void   pciio_pio_write8(uint8_t val, volatile uint8_t *addr);
extern void   pciio_pio_write16(uint16_t val, volatile uint16_t *addr);
extern void   pciio_pio_write32(uint32_t val, volatile uint32_t *addr);
extern void   pciio_pio_write64(uint64_t val, volatile uint64_t *addr);

You must use the pciio_pio_* routines for all PIO access to the device, including accesses to the PCI configuration space. PIO access includes accesses to the device registers (explicit PIO) as well as any memory space that is mapped (implicit PIO). For example, if a device allows access to local memory on the card and the driver maps this memory to the system address space, every access to this address space must be done through the pciio_pio_* routines.

Use the compiler switch -DUSE_PCI_PIO to enable the IP32 PIO read and write routines in the compilation of the IP32 device driver module. Turning this flag on or off lets you use the same source to compile the driver for different target platforms. While the USE_PCI_PIO flag is required for the IP32 architecture, it should not be used when compiling for other architectures. Refer to “Compiling and Linking” in Chapter 9 for details on compiling for different targets.

PCI PIO Code Examples

In the following examples, a function receives kernel virtual addresses (mapped with the pciio_piotrans_addr() call) for a control register and on board memory in the PIO address space of a hypothetical PCI device. The 32-bit control register is set to enable reads of data from on-board memory (accessible as 8-bits).

The function could look like that shown in Example 21-6 for a device driver not written for the O2 platform.

Example 21-6. Non-O2 PCI PIO Code Example

#define ENABLE_SOMETHING 0x1
    void
    example(volatile unsigned int *control_reg,
            volatile unsigned char *device_data,
            int len,
            unsigned char *buffer) 
    {
        int i;
 
        /*
         * set the enable bit
         */ 
        *control_reg |= ENABLE_SOMETHING;
 
        /*
         * copy the data to the caller's buffer
         */
        for (i = 0; i < len; i++)
            *buffer++ = *device_data++;
 
        /*
         * reset enable bit
         */
        *control_reg &= ~ENABLE_SOMETHING;
    }

To work correctly on the O2 platform, the driver code in Example 21-6 would have to change as shown in Example 21-7.

Example 21-7. O2 PCI PIO Code Example

#define ENABLE_SOMETHING 0x1
    void
    example(volatile unsigned int *control_reg,
            volatile unsigned char *device_data,
            unsigned char *buffer, 
            int len)
    {
        int i;
        unsigned int reg_val;
 
        /*
         * NOTE: use of &= and |= for PIO is strongly
         *       discouraged due to the unpredictability
         *       of the actual instructions generated by
         *       the compiler. We recommend breaking
         *       these up into simpler expressions.
         *
         * set the enable bit
         */ 
        reg_val = pciio_pio_read32(control_reg);
        reg_val = reg_val | ENABLE_SOMETHING;
        pciio_pio_write32(reg_val, control_reg);
 
        /*
         * copy the data to the caller's buffer
         */
        for (i = 0; i < len; i++)
            *buffer++ = pciio_pio_read8(device_data++);
        /*
         * reset enable bit
         */
        reg_val = pciio_pio_read32(control_reg);
        reg_val = reg_val & ~ENABLE_SOMETHING;
        pciio_pio_write32(reg_val, control_reg);
    }

If you are writing new code from scratch for multiple platforms, you can use the “O2-style” described here for all supported platforms and just compile for each target platform as described in “Compiling and Linking” in Chapter 9.

Using DMA Maps

You use a DMA map to establish a mapping between a buffer in kernel virtual space and some portion of the PCI bus memory space so that a PCI device can read and write to memory. Depending on the machine architecture, the mapping may be a simple translation function, or it may require the kernel to program hardware registers in one or more bus adapters. The software interface is the same in all cases.

You cannot program a PCI bus master for DMA without at least one DMA map. Often you will allocate two or more. Typically you save the addresses of the allocated maps in the device information structure; and you store the address of the device information structure in turn in the hwgraph vertex for the device.

The functions that are used to manage simple DMA maps are summarized in Table 21-4. Details are found in reference page pciio_dma(d3).

Table 21-4. Functions for Simple DMA Maps for PCI

Function

Header Files

Purpose and Operation

pciio_dmamap_alloc()

ddi.h, pciio.h

Create a DMA map object, specifying the maximum extent of memory the map will have to cover.

pciio_dmamap_addr()

ddi.h, pciio.h

Set up mapping from a kernel memory address for a specified length, to the PCI bus, returning the bus address.

pciio_dmamap_drain()

ddi.h, pciio.h

Complete any active DMA on a specified map. May flush prefetch and gather buffers in the PCI adapter.

pciio_dmamap_list()

ddi.h, pciio.h

Set up a mapping that relates all addresses in an alenlist to the PCI bus, returning a new alenlist containing PCI bus addresses.

pciio_dmalist_drain()

ddi.h, pciio.h

Complete any active DMA on a map set up using pciio_dmamap_list().

pciio_dmamap_done()

ddi.h, pciio.h

Make a DMA map inactive. Release any hardware resources associated to the active mapping.

pciio_dmamap_free()

ddi.h, pciio.h

Release a DMA map object.

pciio_dmatrans_list()

ddi.h, pciio.h

Request immediate translation of the addresses in an alenlist. Returns NULL unless this system supports fixed DMA addressing.

pciio_dmatrans_addr()

ddi.h, pciio.h

Request immediate translation of the address of a contiguous memory buffer to a bus address. Returns NULL unless this system supports fixed DMA addressing.

pcibr_get_dmatrans_node()

ddi.h, pciio.h

Obtain the 32-bit direct mapping node.

pciio_dmaadr_drain()

ddi.h, pciio.h

Complete any active DMA on a mapping established using pciio_dmatrans_addr().

In summary, a DMA map is used as follows:

  1. Allocate it with pciio_dmamap_alloc().

  2. Activate the map and extract a PCI bus memory base address using pciio_dmamap_addr() or pciio_dmamap_list().

  3. Program the base addresses into the PCI bus master device and start the transfer. To start additional transfers that fall in the same mapped segment, repeat this step.

  4. When DMA to the mapped segment is complete, deactivate the map using pciio_dmamap_done().

  5. When further DMA is planned, return to step 2.

  6. When the map is no longer needed, release it with pciio_dmamap_free().

Allocating DMA Maps

A DMA map is created by pciio_dmamap_alloc(), which takes arguments are as follows:

vhdl 

Connection-point vertex_hdl_t received by pfxattach(). This handle identifies the device to the kernel by its bus and slot positions.

dev_desc 

Device descriptor structure (see text following).

byte_count_max 

Maximum size of the area that will be mapped at any one time.

flags 

Usage flags and optional no-sleep flag.


Preparing a device_desc_t

The device descriptor structure type device_desc_t is declared in iobus.h, which is included by pciio.h (see also reference page device_desc(d4x) ). In this release there is little that the device driver needs to know about this structure and its contents. The simplest way to get a device descriptor that can be handed to pciio_dmamap_alloc() is to call device_desc_default_get() passing the same connection-point vertex handle, as follows:

ret = pciio_dmamap_alloc(convh,device_desc_default_get(convh),...)

Setting Flag Values

The following flag values control data transfer:

PCIIO_DMA_CMD

Configure as a generic “command” stream. Generally this means turn off prefetchers and write gatherers, and whatever else might be necessary to make command ring DMAs work as expected.

PCIIO_DMA_DATA

Configure as a generic “data” stream. Generally, this means turning on prefetchers and write gatherers, and anything else that might increase DMA throughput (short of using “high priority” or “real time” resources that may lower overall system performance).

PCIIO_PREFETCH
PCIIO_NOPREFETCH

Control the use of prefetch hardware, overriding the CMD or DATA selection.

PCIIO_WRITE_GATHER
PCIIO_NOWRITE_GATHER

Control the use of write-gather hardware, overriding the CMD or DATA selection.

(Note that the values for the PCIIO_DMA* flags have changed as of IRIX 6.5.) All systems have the ability to gather as much as one cache line of device data before starting a write to memory, but some systems have better write-gather support. All systems have a certain amount of “prefetch” ability in that they load a full cache line from memory when the device issues a read. However, some systems can prefetch the next cache line while the device is still accepting the first one. The PCI infrastructure notes in the DMA map whether you want these features maximized for the given hardware, or minimized, depending on the flag settings.

The following flag values control the use of the PCI bus adapter:

PCIIO_DMA_A64

Device and driver are prepared to use 64-bit addressing.

PCIIO_BYTE_STREAM

Retain the order of a stream of bytes between device and memory.

PCIIO_WORD_VALUES

Byte-swap 32-bit words during transfer as required to produce big-endian data order in memory.

You should specify either PCIIO_BYTE_STREAM or PCIIO_WORD_VALUES; there is no default. (More correctly, the default is “whatever the hardware of this system does,” and different systems do different things.) When you specify PCIIO_BYTE_STREAM, a block of bytes transferred from the device to memory has the same sequence of bytes in both locations. When you specify PCIIO_WORD_VALUES, the numerical significance of the bytes in each 32-bit word are preserved between the big-endian memory and little-endian device.

The following flag values control kernel operations:

PCIIO_NOSLEEP

Do not sleep on memory allocation.

PCIIO_INPLACE

Translate alenlists in place instead of copying them.


Using a DMA Map

After a map has been allocated, it is inactive. When you apply a function to a map to get a translated address, the function activates the map if it is not active, and uses the map to set up a correspondence between PCI bus memory addresses and one or more segments of kernel virtual address space.

In some systems, “activating a map” can be a null operation. In other systems, an active DMA map may represent a commitment of limited hardware resources—for example, mapping registers in a bus adapter.

You can use a DMA map to map a specified memory segment, or you can use it to translate all entries in an address/length list (see “Address/Length Lists” in Chapter 8) in a single operation.

Mapping an Address/Length List

You map an alenlist using pciio_dmamap_list(). This function takes an alenlist that represents a memory buffer, and in one operation produces a new list containing PCI bus addresses and lengths. You read out the translated addresses from the list and program them into the bus master device (see “Using Address/Length Lists” in Chapter 8).

Mapping a Specific Buffer

You obtain a DMA map for a single, contiguous span of kernel virtual memory by calling pciio_dmamap_addr(). If the mapping cannot be set up, 0 is returned. (You must check for this possibility, because if you start a DMA transfer to location 0, a bus error results.) Otherwise the value returned is a PCI bus address that you can program into a bus master device. When the device accesses that address, it accesses the specified memory location.

Completing DMA Transfers

If it is necessary to establish that a DMA transfer is fully complete—all input data stored in physical memory, all output data copied from memory—use the “drain” function that corresponds to the way the map was activated. For example, if the map was activated using pciio_dmamap_list(), you call pciio_dmalist_drain() to ensure that current DMA is complete. When the bus adapter uses prefetch buffers or write-gather buffers, they are flushed.

Deactivating Addresses and Maps

Once you have created a mapping, the map is active. It remains active until you use the same DMA map object to map a different buffer, or until you call either pciio_dmamap_done() or pciio_dmamap_free().

In some systems, it costs nothing to keep a DMA map active. In other systems, an active map may tie up global hardware resources. It is a good idea to call pciio_dmamap_done() when the I/O operation is complete.


Caution: Never call pciio_dmamap_done() before the device has stopped sending data. Memory corruption could result.


Using One-Step DMA Translation

Some systems also support one-step mapping functions pciio_dmatrans_addr() and pciio_dmatrans_list(). In effect, these functions combine creating a map, using the map, and freeing the map into a single step. They can fail (returning 0) in systems that do not use simple bus maps. If you use them, you must test the returned address. If it is 0, the one-step translation failed and the address is invalid.

If the PCI device supports only 32-bit addresses, the one-step mapping functions map the PCI address space to system memory on one specific node. This means that any memory (DMA buffers) that you want to map must be allocated on that node. The default is node zero; however, the node can be changed for any PCI bus (except on O2 and Octane systems, where the node is zero and cannot be changed) by using the DEVICE_ADMIN statement (see “Setting 32-bit Direct Mapping Node” in Chapter 2). You can use the pcibr_get_dmatrans_node() function to obtain the node that is being used by a specific PCI bus.

The two-step process of allocating a map and then interrogating it is more general and works in all systems.

Interrogating DMA Maps

The following functions can be used to interrogate a DMA map object (see pciio_get(d3) ):

Table 21-5. Functions for Interrogating DMA Maps

Function

Header Files

Purpose and Operation

pciio_dma_dev_get()

ddi.h, pciio.h

Return the connection point handle from a map.

pciio_dma_slot_get()

ddi.h, pciio.h

Return the bus slot number of the mapped device.


Registering an Interrupt Handler

When a device can interrupt, you must register an interrupt handler for it. This is done in a two-step process. First you create an interrupt connection object; then you use that object to specify the interrupt handling function. Prior to unloading the driver or detaching the device, you must unregister the handler (but you can retain the interrupt connection object).

The functions for managing interrupt handlers are summarized in Table 21-6. For syntax details see reference page pciio_intr(d3).

Table 21-6. Functions for Managing PCI Interrupt Handlers

Function

Purpose and Operation

pciio_intr_alloc()

Create an interrupt object that enables interrupts to flow from a specified device.

pciio_intr_connect()

Associate an interrupt object with an interrupt handler function.

pciio_intr_disconnect()

Remove the association between an interrupt object and a handler function.

pciio_intr_free()

Release an interrupt object.


Creating an Interrupt Object

A software object that represents an interrupt connection is created with pciio_intr_alloc(), which takes the following arguments:

vhdl 

The hwgraph vertex for the device attachment point—the same vertex originally passed to the pfxattach() entry point.

desc 

Device descriptor structure (see text following).

lines 

The selection of one or more PCI interrupt lines used by this device, a sum of PCIIO_INTR_LINE_A, PCIIO_INTR_LINE_B, ..._C, and ..._D.

owner 

The hwgraph vertex to use when reporting errors—same as vhdl, or else a vertex created by the driver.

The interrupt object is used in establishing a handler, and it is needed later to stop taking interrupts. You should save its address in the device information structure you store in the hwgraph vertex.

Preparing a device_desc_t

The device descriptor structure type device_desc_t is declared in iobus.h, which is included by pciio.h (see also reference page device_desc(d4x) ). In this release there is little that the device driver needs to know about this structure and its contents. The simplest way to get a device descriptor that can be handed to pciio_intr_alloc() is to call device_desc_default_get() passing the same connection-point vertex handle, as follows:

ret = pciio_intr_alloc(convh,device_desc_default_get(convh),...)

Connecting the Handler

After creating the interrupt object, you establish a handler using pciio_intr_connect(). Its principal arguments are the interrupt object, a handler address, and a value to be passed to the handler when it is called:

intr 

The value returned by pciio_intr_alloc().

func 

Address of the function to be called when an interrupt occurs; see following text for the prototype.

arg 

A pointer-sized value to be passed as the argument to func each time it is called. Typically the address of the device information structure, or the handle of the device vertex.

thread 

Passed as NULL.

Before calling pciio_intr_connect(), an interrupt handler should call device_desc_dup(), device_desc_intr_name_set(), and device_desc_default_set() in that order. See the device_desc_ops(D3X) reference page for more information.

Connecting the Handler

The function prototype for the interrupt handler is named intr_func_t and is declared in sys/iobus.h (which is included by sys/PCI/pciio.h). The function prototype is

typedef void         *intr_arg_t;
typedef void         intr_func_f(intr_arg_t);
typedef intr_func_f  *intr_func_t;


Caution: This function prototype differs from the same prototype as it was declared in IRIX 6.3. The interrupt handler in 6.3 is declared to receive one additional argument that is not supported in the later releases.

If a device will interrupt on line C, interrupt setup could resemble Example 21-8.

Example 21-8. Setting Up a PCI Interrupt Handler

pciio_intr_t intobj;
extern void int_handler(devinfo*);
int retcode;
device_desc_t dev_desc;
intobj = pciio_intr_alloc(
               vhdl, /* as received in attach() */
               0,    /* device descriptor is n.a. for pci */
               PCIIO_INTR_LINE_C, /* the line it uses */
               vhdl);
dev_desc = device_desc_dup(vhdl);
device_desc_intr_name_set(0, "PCI");
device_desc_default_set(vhdl, 0);
retcode = pciio_intr_connect(
               intobj, /* the interrupt object */
               (intr_func_t) int_handler, /* the handler */
               (intr_arg_t) pDevInfo, /* dev info as input */
               (void*)0 ); /* let kernel pick the thread */
if (!retcode) cmn_err(CE_WARN,"oh fiddlesticks");


Handler Operation

Interrupts from a device are disabled (if possible) and discarded until a handler is connected. However, interrupts in general are enabled when the pfxattach() entry point is called. If the PCI device is in a state that can produce an interrupt, the interrupt handling function can be called before pciio_intr_connect() returns. Make sure that all global data used by the interrupt handler has been initialized before you connect it.

When called, the interrupt handler runs as an independent thread in the kernel. It can run concurrently with any other part of the driver, and concurrently with other interrupt handlers. Although interrupt threads run at a high priority, there are kernel threads with still higher priority that can preempt the interrupt handler. See “Interrupt Entry Point and Handler” in Chapter 7.

PCI devices can share the four PCI interrupt lines. As a result, in some cases the kernel cannot tell which device caused an interrupt. When there is any doubt, the kernel calls all the interrupt handlers that are registered to that interrupt line. For this reason, your interrupt handler must not assume that its device did cause the interrupt. It should always test to see if an interrupt is really pending, and exit immediately when one is not.

The handler gets information about the device only from the argument that was passed to it. This is presumably the device information structure, containing driver-specific information about the device and its status, PIO maps, and the vertex handle of the connection point as passed to pfxattach(). This handle can be used to get more information about the device; see “Interrogating a PCI Device”.


Note: Lost interrupts can occur. An application must not set the interrupt line if it is already set, because this causes a race condition. Furthermore, an application must wait a small amount of time after the interrupt line transitions from set to unset, to make sure the minimum deassert time is not violated. One a solution has been to modify firmware never to set the interrupt line unless the host has cleared it, and also wait a few clocks.


Disconnecting the Handler

The only way to stop receiving interrupts is to disconnect the handler. This is done with a call to pciio_intr_disconnect(). Its only argument is the interrupt object returned by pciio_intr_alloc().

Interrogating an Interrupt Handler

The following functions can be used to interrogate an interrupt object (see pciio_get(d3) ):

Table 21-7. Functions for Interrogating an Interrupt Object

Function

Header Files

Purpose and Operation

pciio_intr_dev_get()

ddi.h, pciio.h

Return the connection point handle from the object.

pciio_intr_cpu_get()

ddi.h, pciio.h

Return the CPU that receives the hardware interrupt.


Registering an Error Handler

You can register a function to be called in case of a bus error related to a specific PCI device. When the kernel detects a bus error, and can isolate the error to the bus address space related to one of your PIO or DMA maps, it calls the error handling function. If the function can correct the error, it returns 0. If it cannot, or if it does not understand the error, it returns 1, and the kernel continues with default error actions.

The declarations used to set up an error handler are summarized in Table 21-8 (see also reference page pciio_error(d3)).

Table 21-8. Declaration Used In Setting Up PCI Error Handlers

Identifier

Header File

Purpose or Use

ioerror_mode_t 

sys/ioerror. h 

Enumeration for the kernel mode during which the error was found: probing the bus, normal operations, user-mode access, or error retry.

ioerror_t 

sys/ioerror. h 

Structure giving details of an error.

error_handler_arg_t 

sys/ioerror. h 

Name for void*, the opaque value provided by the driver to be passed to the error handler to describe the device.

error_handler_f 

sys/ioerror. h 

Name for the prototype of an error handler function, convenient for forward or extern declaration of the handler.

pciio_error_register()

sys/PCI/pcii o.h 

Function to register or unregister an error handler.

You code your error handler using the prototype established by error_handler_f:

typedef int
error_handler_f(
      error_handler_arg_t arg, /* device info, registered */
      int error_code,            /* IOEC_* values in sys/ioerror.h */
      ioerror_mode_t mode,    /* mode value in sys/ioerror.h */
      ioerror_t *info);

You register the handler by calling pciio_error_register(), passing three values:

  • The vertex handle of the connection point, as passed to the pfxattach() entry point).

  • The address of the error handler function.

  • An address to be passed as the first argument of the error handler function, when it is called.

To unregister the error handler (when the driver is unloading, or when detaching the device), call pciio_error_register() with the same vertex handle, but with NULL for the address of the handler. 

Interrogating a PCI Device

The following functions can be used to get an information structure that describes a PCI device, and to extract data from it (see pciio_get(d3) ):

Table 21-9. Functions for Interrogating a PCI Device

Function

Header Files

Purpose and Operation

pciio_info_get()

ddi.h, pciio.h

Given the connection point as passed to pfxattach(), return a read-only object with information about the device.

pciio_info_dev_get()

ddi.h, pciio.h

Return the connection point handle.

pciio_info_bus_get()

ddi.h, pciio.h

Return the PCI bus number.

pciio_info_slot_get()

ddi.h, pciio.h

Return the slot number on the bus.

pciio_info_func_get()

ddi.h, pciio.h

Return the function number of the device (a PCI card can have up to 8 separate functions).

pciio_info_vendor_id_get()

ddi.h, pciio.h

Return the vendor ID value.

pciio_info_device_id_get()

ddi.h, pciio.h

Return the device ID value.


Example PCI Driver

/**********************************************************************
*              Copyright (C) 1990-1993, Silicon Graphics, Inc.        *
* These coded instructions, statements, and computer programs contain *
* unpublished proprietary information of Silicon Graphics, Inc., and  *
* are protected by Federal copyright law.  They may not be disclosed  *
* to third parties or copied or duplicated in any form, in whole or   *
* in part, without prior written consent of Silicon Graphics, Inc.    *
**********************************************************************/
#ident  "io/sample_pciio.c: $Revision: 1.17 $"
#include <sys/types.h>
#include <sys/cpu.h>
#include <sys/systm.h>
#include <sys/cmn_err.h>
#include <sys/errno.h>
#include <sys/buf.h>
#include <sys/ioctl.h>
#include <sys/cred.h>
#include <ksys/ddmap.h>
#include <sys/poll.h>
#include <sys/invent.h>
#include <sys/debug.h>
#include <sys/sbd.h>
#include <sys/kmem.h>
#include <sys/edt.h>
#include <sys/dmamap.h>
#include <sys/hwgraph.h>
#include <sys/iobus.h>
#include <sys/iograph.h>
#include <sys/param.h>
#include <sys/pio.h>
#include <sys/sema.h>
#include <sys/ddi.h>
#include <sys/atomic_ops.h>
#include <sys/PCI/PCI_defs.h>
#include <sys/PCI/pciio.h>
#define NEW(ptr)        (ptr = kmem_alloc(sizeof (*(ptr)), KM_SLEEP))
#define DEL(ptr)        (kmem_free(ptr, sizeof (*(ptr))))
/*
 *    psamp: a generic device driver for a generic PCI device.
 */
int         psamp_devflag = D_MP;
int         psamp_inuse = 0;     /* number of "psamp" devices open */
/* ====================================================================
 *          Device-Related Constants and Structures
 */
#define PSAMP_VENDOR_ID_NUM     0x5555
#define PSAMP_DEVICE_ID_NUM     0x555
/*
 *    All registers on the Sample PCIIO Client
 *      device are 32 bits wide.
 */
typedef __uint32_t      psamp_reg_t;
typedef volatile struct psamp_regs_s *psamp_regs_t; /* dev registers */
typedef struct psamp_soft_s *psamp_soft_t;         /* software state */
/*
 *    psamp_regs: layout of device registers
 *      Our device config registers are, of course, at
 *      the base of our assigned CFG space.
 *      Our sample device registers are in the PCI area
 *      decoded by the device's first BASE_ADDR window.
 */
struct psamp_regs_s {
    psamp_reg_t             pr_control;
    psamp_reg_t             pr_status;
};
struct psamp_soft_s {
    vertex_hdl_t      ps_conn;    /* connection for pci services */
    vertex_hdl_t      ps_vhdl;    /* backpointer to device vertex */
    vertex_hdl_t      ps_blockv;  /* backpointer to block vertex */
    vertex_hdl_t      ps_charv;   /* backpointer to char vertex */
    volatile uchar_t  ps_cfg;     /* cached ptr to my config regs */
    psamp_regs_t      ps_regs;    /* cached ptr to my regs */
    pciio_piomap_t    ps_cmap;    /* piomap (if any) for ps_cfg */
    pciio_piomap_t    ps_rmap;    /* piomap (if any) for ps_regs */
    unsigned          ps_sst;     /* driver "software state" */
#define PSAMP_SST_RX_READY      (0x0001)
#define PSAMP_SST_TX_READY      (0x0002)
#define PSAMP_SST_ERROR         (0x0004)
#define PSAMP_SST_INUSE         (0x8000)
    pciio_intr_t      ps_intr;       /* pciio intr for INTA and INTB */
    pciio_dmamap_t    ps_ctl_dmamap; /* control channel dma mapping */
    pciio_dmamap_t    ps_str_dmamap; /* stream channel dma mapping */
    struct pollhead   *ps_pollhead;  /* for poll() */
    int               ps_blocks;  /* block dev size in NBPSCTR blocks */
};
#define psamp_soft_set(v,i)     device_info_set((v),(void *)(i))
#define psamp_soft_get(v)       ((psamp_soft_t)device_info_get((v)))
/*=====================================================================
 *          FUNCTION TABLE OF CONTENTS
 */
void                  psamp_init(void);
int                   psamp_unload(void);
int                   psamp_reg(void);
int                   psamp_unreg(void);
int                   psamp_attach(vertex_hdl_t conn);
int                   psamp_detach(vertex_hdl_t conn);
static pciio_iter_f   psamp_reloadme;
static pciio_iter_f   psamp_unloadme;
int                   psamp_open(dev_t *devp, int oflag, int otyp,
                                 cred_t *crp);
int                   psamp_close(dev_t dev, int oflag, int otyp,
                                  cred_t *crp);
int                   psamp_ioctl(dev_t dev, int cmd, void *arg,
                                  int mode, cred_t *crp, int *rvalp);
int                   psamp_read(dev_t dev, uio_t * uiop, cred_t *crp);
int                   psamp_write(dev_t dev, uio_t * uiop,cred_t *crp);
int                   psamp_strategy(struct buf *bp);
int                   psamp_poll(dev_t dev, short events, int anyyet,
                               short *reventsp, struct pollhead **phpp,
                               unsigned int *genp);
int                   psamp_map(dev_t dev, vhandl_t *vt,
                                off_t off, size_t len, uint_t prot);
int                   psamp_unmap(dev_t dev, vhandl_t *vt);
void                  psamp_dma_intr(intr_arg_t arg);
static error_handler_f psamp_error_handler;
void                  psamp_halt(void);
int                   psamp_size(dev_t dev);
int                   psamp_print(dev_t dev, char *str);
 
/*=====================================================================
 *                  Driver Initialization
 */
/*
 *    psamp_init: called once during system startup or
 *      when a loadable driver is loaded.
 */
void
psamp_init(void)
{
    printf("psamp_init()\n");
    /*
     * if we are already registered, note that this is a
     * "reload" and reconnect all the places we attached.
     */
    pciio_iterate("psamp_", psamp_reloadme);
}
/*
 *    psamp_unload: if no "psamp" is open, put us to bed
 *      and let the driver text get unloaded.
 */
int
psamp_unload(void)
{
    if (psamp_inuse)
        return EBUSY;
    pciio_iterate("psamp_", psamp_unloadme);
    return 0;
}
/*
 *    psamp_reg: called once during system startup or
 *      when a loadable driver is loaded.
 *    NOTE: a bus provider register routine should always be
 *      called from _reg, rather than from _init. In the case
 *      of a loadable module, the devsw is not hooked up
 *      when the _init routines are called.
 */
int
psamp_reg(void)
{
    printf("psamp_reg()\n");
    pciio_driver_register(PSAMP_VENDOR_ID_NUM,
                          PSAMP_DEVICE_ID_NUM,
                          "psamp_",
                          0);
    return 0;
}
/*
 *    psamp_unreg: called when a loadable driver is unloaded.
 */
int
psamp_unreg(void)
{
    pciio_driver_unregister("psamp_");
    return 0;
}
/*
 *    psamp_attach: called by the pciio infrastructure
 *      once for each vertex representing a crosstalk widget.
 *      In large configurations, it is possible for a
 *      huge number of CPUs to enter this routine all at
 *      nearly the same time, for different specific
 *      instances of the device. Attempting to give your
 *      devices sequence numbers based on the order they
 *      are found in the system is not only futile but may be
 *      dangerous as the order may differ from run to run.
 */
int
psamp_attach(vertex_hdl_t conn)
{
    vertex_hdl_t            vhdl, blockv, charv;
    volatile uchar_t       *cfg;
    psamp_regs_t            regs;
    psamp_soft_t            soft;
    pciio_piomap_t          cmap = 0;
    pciio_piomap_t          rmap = 0;
    printf("psamp_attach()\n");
    hwgraph_device_add(conn,"psamp","psamp_",&vhdl,&blockv,&charv);
    /*
     * Allocate a place to put per-device information for this vertex.
     * Then associate it with the vertex in the most efficient manner.
     */
    NEW(soft);
    ASSERT(soft != NULL);
    psamp_soft_set(vhdl, soft);
    psamp_soft_set(blockv, soft);
    psamp_soft_set(charv, soft);
    soft->ps_conn = conn;
    soft->ps_vhdl = vhdl;
    soft->ps_blockv = blockv;
    soft->ps_charv = charv;
    /*
     * Find our PCI CONFIG registers.
     */
    cfg = (volatile uchar_t *) pciio_pio_addr
        (conn, 0,               /* device and (override) dev_info */
         PCIIO_SPACE_CFG,       /* select configuration addr space */
         0,                     /* from the start of space, */
         PCI_CFG_VEND_SPECIFIC, /* ... up to vendor specific stuff */
         &cmap,                 /* in case we needed a piomap */
         0);                    /* flag word */
    soft->ps_cfg = cfg;         /* save for later */
    soft->ps_cmap = cmap;
    printf("psamp_attach: I can see my CFG regs at 0x%x\n", cfg);
    /*
     * Get a pointer to our DEVICE registers
     */
    regs = (psamp_regs_t) pciio_pio_addr
        (conn, 0,               /* device and (override) dev_info */
         PCIIO_SPACE_WIN(0),    /* in my primary decode window, */
         0, sizeof(*regs),      /* base and size */
         &rmap,                 /* in case we needed a piomap */
         0);                    /* flag word */
    soft->ps_regs = regs;       /* save for later */
    soft->ps_rmap = rmap;
    printf("psamp_attach: I can see my device regs at 0x%x\n", regs);
    /*
     * Set up our interrupt.
     * We might interrupt on INTA or INTB,
     * but route 'em both to the same function.
     */
    soft->ps_intr = pciio_intr_alloc
        (conn, 0,
         PCIIO_INTR_LINE_A |
         PCIIO_INTR_LINE_B,
         vhdl);
    pciio_intr_connect(soft->ps_intr,
                       psamp_dma_intr, soft,(void *) 0);
    /*
     * set up our error handler.
     */
    pciio_error_register(conn, psamp_error_handler, soft);
    /*
     * For pciio clients, *now* is the time to
     * allocate pollhead structures.
     */
    soft->ps_pollhead = phalloc(0);
    return 0;                      /* attach successsful */
}
/*
 *    psamp_detach: called by the pciio infrastructure
 *      once for each vertex representing a crosstalk
 *      widget when unregistering the driver.
 *
 *      In large configurations, it is possible for a
 *      huge number of CPUs to enter this routine all at
 *      nearly the same time, for different specific
 *      instances of the device. Attempting to give your
 *      devices sequence numbers based on the order they
 *      are found in the system is not only futile but may be
 *      dangerous as the order may differ from run to run.
 */
int
psamp_detach(vertex_hdl_t conn)
{
    vertex_hdl_t            vhdl, blockv, charv;
    psamp_soft_t            soft;
    printf("psamp_detach()\n");
    if (GRAPH_SUCCESS !=
        hwgraph_traverse(conn, "psamp", &vhdl))
        return -1;
    soft = psamp_soft_get(vhdl);
    pciio_error_register(conn, 0, 0);
    pciio_intr_disconnect(soft->ps_intr);
    pciio_intr_free(soft->ps_intr);
    phfree(soft->ps_pollhead);
    if (soft->ps_ctl_dmamap)
        pciio_dmamap_free(soft->ps_ctl_dmamap);
    if (soft->ps_str_dmamap)
        pciio_dmamap_free(soft->ps_str_dmamap);
    if (soft->ps_cmap)
        pciio_piomap_free(soft->ps_cmap);
    if (soft->ps_rmap)
        pciio_piomap_free(soft->ps_rmap);
    hwgraph_edge_remove(conn, "psamp", &vhdl);
    /*
     * we really need "hwgraph_dev_remove" ...
     */
    if (GRAPH_SUCCESS ==
        hwgraph_edge_remove(vhdl, EDGE_LBL_BLOCK, &blockv)) {
        psamp_soft_set(blockv, 0);
        hwgraph_vertex_destroy(blockv);
    }
    if (GRAPH_SUCCESS ==
        hwgraph_edge_remove(vhdl, EDGE_LBL_CHAR, &charv)) {
        psamp_soft_set(charv, 0);
        hwgraph_vertex_destroy(charv);
    }
    psamp_soft_set(vhdl, 0);
    hwgraph_vertex_destroy(vhdl);
    DEL(soft);
    return 0;
}
/*
 *    psamp_reloadme: utility function used indirectly
 *      by psamp_init, via pciio_iterate, to "reconnect"
 *      each connection point when the driver has been
 *      reloaded.
 */
static void
psamp_reloadme(vertex_hdl_t conn)
{
    vertex_hdl_t            vhdl;
    psamp_soft_t            soft;
    if (GRAPH_SUCCESS !=
        hwgraph_traverse(conn, "psamp", &vhdl))
        return;
    soft = psamp_soft_get(vhdl);
    /*
     * Reconnect our error and interrupt handlers
     */
    pciio_error_register(conn, psamp_error_handler, soft);
    pciio_intr_connect(soft->ps_intr, psamp_dma_intr, soft, 0);
}
/*
 *    psamp_unloadme: utility function used indirectly by
 *      psamp_unload, via pciio_iterate, to "disconnect" each
 *      connection point before the driver becomes unloaded.
 */
static void
psamp_unloadme(vertex_hdl_t pconn)
{
    vertex_hdl_t            vhdl;
    psamp_soft_t            soft;
    if (GRAPH_SUCCESS !=
        hwgraph_traverse(pconn, "psamp", &vhdl))
        return;
    soft = psamp_soft_get(vhdl);
    /*
     * Disconnect our error and interrupt handlers
     */
    pciio_error_register(pconn, 0, 0);
    pciio_intr_disconnect(soft->ps_intr);
}
/* ====================================================================
 *          DRIVER OPEN/CLOSE
 */
/*
 *    psamp_open: called when a device special file is
 *      opened or when a block device is mounted.
 */
/* ARGSUSED */
int
psamp_open(dev_t *devp, int oflag, int otyp, cred_t *crp)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(*devp);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    psamp_regs_t            regs = soft->ps_regs;
    printf("psamp_open() regs=%x\n", regs);
    /*
     * BLOCK DEVICES: now would be a good time to
     * calculate the size of the device and stash it
     * away for use by psamp_size.
     */
    /*
     * USER ABI (64-bit): chances are, you are being
     * compiled for use in a 64-bit IRIX kernel; if
     * you use the _ioctl or _poll entry points, now
     * would be a good time to test and save the
     * user process' model so you know how to
     * interpret the user ioctl and poll requests.
     */
    if (!(PSAMP_SST_INUSE & atomicSetUint(&soft->ps_sst, PSAMP_SST_INUSE)))
        atomicAddInt(&psamp_inuse, 1);
    return 0;
}
/*
 *    psamp_close: called when a device special file
 *      is closed by a process and no other processes
 *      still have it open ("last close").
 */
/* ARGSUSED */
int
psamp_close(dev_t dev, int oflag, int otyp, cred_t *crp)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(dev);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    psamp_regs_t            regs = soft->ps_regs;
    printf("psamp_close() regs=%x\n", regs);
    atomicClearUint(&soft->ps_sst, PSAMP_SST_INUSE);
    atomicAddInt(&psamp_inuse, -1);
    return 0;
}
/* ====================================================================
 *          CONTROL ENTRY POINT
 */
/*
 *    psamp_ioctl: a user has made an ioctl request
 *      for an open character device.
 *      Arguments cmd and arg are as specified by the user;
 *      arg is probably a pointer to something in the user's
 *      address space, so you need to use copyin() to
 *      read through it and copyout() to write through it.
 */
/* ARGSUSED */
int
psamp_ioctl(dev_t dev, int cmd, void *arg,
            int mode, cred_t *crp, int *rvalp)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(dev);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    psamp_regs_t            regs = soft->ps_regs;
    printf("psamp_ioctl() regs=%x\n", regs);
    *rvalp = -1;
    return ENOTTY;          /* TeleType® is a registered trademark */
}
/* ====================================================================
 *          DATA TRANSFER ENTRY POINTS
 *      Since I'm trying to provide an example for both
 *      character and block devices, I'm routing read
 *      and write back through strategy as described in
 *      the IRIX Device Driver Programming Guide.
 *      This limits our character driver to reading and
 *      writing in multiples of the standard sector length.
 */
/* ARGSUSED */
int
psamp_read(dev_t dev, uio_t * uiop, cred_t *crp)
{
    return physiock(psamp_strategy,
                    0,          /* alocate temp buffer & buf_t */
                    dev,        /* dev_t arg for strategy */
                    B_READ,     /* direction flag for buf_t */
                    psamp_size(dev),
                    uiop);
}
/* ARGSUSED */
int
psamp_write(dev_t dev, uio_t * uiop, cred_t *crp)
{
    return physiock(psamp_strategy,
                    0,          /* alocate temp buffer & buf_t */
                    dev,        /* dev_t arg for strategy */
                    B_WRITE,    /* direction flag for buf_t */
                    psamp_size(dev),
                    uiop);
}
/* ARGSUSED */
int
psamp_strategy(struct buf *bp)
{
    /*
     * XXX - create strategy code here.
     */
    return 0;
}
/* ====================================================================
 *          POLL ENTRY POINT
 */
int
psamp_poll(dev_t dev, short events, int anyyet,
           short *reventsp, struct pollhead **phpp, unsigned int *genp)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(dev);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    psamp_regs_t            regs = soft->ps_regs;
    short                   happened = 0;
    unsigned int            gen;
    printf("psamp_poll() regs=%x\n", regs);
    /*
     * Need to snapshot the pollhead generation number before we check
     * device state.  In many drivers a lock is used to interlock the
     * "high" and "low" portions of the driver.  In those cases we can
     * wait to do this snapshot till we're in the critical region.
     * Snapshotting it early isn't a problem since that makes the
     * snapshotted generation number a more conservative estimate of
     * what generation of  pollhead our event state report indicates.
     */
    gen = POLLGEN(soft->ps_pollhead);
    if (events & (POLLIN | POLLRDNORM))
        if (soft->ps_sst & PSAMP_SST_RX_READY)
            happened |= POLLIN | POLLRDNORM;
    if (events & POLLOUT)
        if (soft->ps_sst & PSAMP_SST_TX_READY)
            happened |= POLLOUT;
    if (soft->ps_sst & PSAMP_SST_ERROR)
        happened |= POLLERR;
    *reventsp = happened;
    if (!happened && anyyet) {
        *phpp = soft->ps_pollhead;
        *genp = gen;
    }
    return 0;
}
/* ====================================================================
 *          MEMORY MAP ENTRY POINTS
 */
/* ARGSUSED */
int
psamp_map(dev_t dev, vhandl_t *vt,
          off_t off, size_t len, uint_t prot)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(dev);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    vertex_hdl_t            conn = soft->ps_conn;
    psamp_regs_t            regs = soft->ps_regs;
    pciio_piomap_t          amap = 0;
    caddr_t                 kaddr;
    printf("psamp_map() regs=%x\n", regs);
    /*
     * Stuff we want users to mmap is in our second BASE_ADDR window.
     */
    kaddr = (caddr_t) pciio_pio_addr
        (conn, 0,
         PCIIO_SPACE_WIN(1),
         off, len, &amap, 0);
    if (kaddr == NULL)
        return EINVAL;
    /*
     * XXX - must stash amap somewhere so we can pciio_piomap_free it
     * when the mapping goes away.
     */
    v_mapphys(vt, kaddr, len);
    return 0;
}
/* ARGSUSED2 */
int
psamp_unmap(dev_t dev, vhandl_t *vt)
{
    /*
     * XXX - need to find "amap" that we used in psamp_map() above,
     * and if (amap) pciio_piomap_free(amap);
     */
    return (0);
}
/* ====================================================================
 *          INTERRUPT ENTRY POINTS
 *  We avoid using the standard name, since our prototype has changed.
 */
void
psamp_dma_intr(intr_arg_t arg)
{
    psamp_soft_t            soft = (psamp_soft_t) arg;
    vertex_hdl_t            vhdl = soft->ps_vhdl;
    psamp_regs_t            regs = soft->ps_regs;
    cmn_err(CE_CONT, "psamp %v: dma done, regs at 0x%X\n", vhdl, regs);
    /*
     * for each buf our hardware has processed,
     *      set buf->b_resid,
     *      call pciio_dmamap_done,
     *      call bioerror() or biodone().
     *
     * XXX - would it be better for buf->b_iodone
     * to be used to get to pciio_dmamap_done?
     */
    /*
     * may want to call pollwakeup.
     */
}
/* ====================================================================
 *          ERROR HANDLING ENTRY POINTS
 */
static int
psamp_error_handler(void *einfo,
                    int error_code,
                    ioerror_mode_t mode,
                    ioerror_t *ioerror)
{
    psamp_soft_t            soft = (psamp_soft_t) einfo;
    vertex_hdl_t            vhdl = soft->ps_vhdl;
#if DEBUG && ERROR_DEBUG
    cmn_err(CE_CONT, "%v: psamp_error_handler\n", vhdl);
#else
    vhdl = vhdl;
#endif
    /*
     * XXX- there is probably a lot more to do
     * to recover from an error on a real device;
     * experts on this are encouraged to add common
     * things that need to be done into this function.
     */
    ioerror_dump("sample_pciio", error_code, mode, ioerror);
    return IOERROR_HANDLED;
}
/* ====================================================================
 *          SUPPORT ENTRY POINTS
 */
/*
 *    psamp_halt: called during orderly system
 *      shutdown; no other device driver call will be
 *      made after this one.
 */
void
psamp_halt(void)
{
    printf("psamp_halt()\n");
}
/*
 *    psamp_size: return the size of the device in
 *      "sector" units (multiples of NBPSCTR).
 */
int
psamp_size(dev_t dev)
{
    vertex_hdl_t            vhdl = dev_to_vhdl(dev);
    psamp_soft_t            soft = psamp_soft_get(vhdl);
    return soft->ps_blocks;
}
/*
 *    psamp_print: used by the kernel to report an
 *      error detected on a block device.
 */
int
psamp_print(dev_t dev, char *str)
{
    cmn_err(CE_NOTE, "%V: %s\n", dev, str);
    return 0;
}

Other Code Examples

The Developer's Toolkit CD-ROM contains a sample PCI device driver for the Barco Chameleon Color Converter. Look in the toolbox/hardware/PCI/barco directory.