The C–brick performs the computing function for the server system; it contains two or four processors (each with 4 or 8 MB of private secondary cache), memory, and a crossbar switch used as a memory controller.

This switch acts as a channel between the internal processors and local memory. The R–brick (router) is used to connect to other C–bricks. The I/O interface bricks (I–brick, P–brick, X–brick) connect to individual C–bricks.

A D–brick can be optionally added to your server. The D–brick is used to provide storage modules to your server.

The G–brick or a V–brick (but not both on the same server system) can also be optionally installed onto your server to provide your system with sophisticated graphics capabilities. The N–brick can be used in systems with a G–brick to replace up to four I– or X–bricks to connect C–bricks with the InfiniteReality graphics pipes on the G–brick. See SGI Onyx 3000 Series Graphics System Hardware Owner's Guide for details on these bricks.

See Figure 1-2 for an illustration of how the bricks are interconnected to create your server system. (For illustrations of how these various bricks are cabled together, see Chapter 4, “C–brick”.)

Figure 1-2. Interaction between SGI Origin 3000 Server Series Bricks

The bricks are air-cooled devices; airflow is from the front of the brick to the rear. All bricks have three fans at the front, except for the R–brick, which has two fans.

The fans run at variable speeds; the speed is controlled by the brick's L1 controller, which monitors and controls the brick's operating temperature. The R–brick fans are smaller than other brick fans and run at a single speed.

All fans are N+1 redundant and can be hot-swapped by a qualified SGI system support engineer (SSE). If a fan fails, the remaining functional fans run at higher speeds to compensate, and error messages are issued.

The C–brick, R–brick, I–brick, P–brick, and X–brick have L1 controllers that monitor the activities of their brick. The L1 controller generates status and error messages for the brick that are displayed on the L1 controller display located on the brick's front panel.

The L2 controller comes with all tall racks with C–bricks (L2 controller is optional for the short racks). The L2 controller in tall racks has a touch display unit located on the front door of the rack. The L2 controller displays system controller status and error messages. It displays the status and error messages generated by the L1 controllers.

The L1 controller messages can also be displayed in a server system console connected to your server system. You can also enter L1 and L2 commands at your console to control activity in your server system.

For details about the L1 and L2 controllers, see Chapter 3, “System Control ”. For a list of L1 and L2 commands, see Appendix B, “System Controller Commands”.

The D–brick has an ESI/ops panel module with a microcontroller for monitoring and controlling all elements of the D–brick.

See SGI Onyx 3000 Series Graphics System Hardware Owner's Guide for details on monitoring and controlling the activity of the G–brick, V–brick, and N–brick.

The short rack (shown in Figure 1-3) has the following features and components:

The tall rack shown in Figure 1-4 has the following features and components:

The racks are measured in EIA standard units; one SU or unit (U) is equal to 1.75 in. (4.49 cm).

Figure 1-5 illustrates the size in standard units of each brick type available with an SGI Origin 3000 series server, except for the G–brick, V–brick, and N–brick. The G–brick is 18 units high, the V–brick four units high, and the N–brick two units high. If you have a G–brick or a V–brick, or a G–brick with an N–brick, see SGI Onyx 3000 Series Graphics System Hardware Owner's Guide for details.

Figure 1-5. Rack and Brick Measurements

Bays in the racks are numbered using standard units, as shown in Figure 1-5. The illustration also identifies the size (in units) of the different brick types.

Because bricks require multiple standard units to fit in a rack, brick locations within a rack are identified by the bottom unit (U) in which the brick resides. For example, in a tall 39U rack, the C–brick positioned in U10, U11, and U12 is identified as C10. In a short 17U rack, an I–brick positioned in U10, U11, U12, and U13 is identified as I10.

These identifiers or bay locations are represented as decimal numbers in the hardware graph path. The hardware graph path has the following form:

/hw/module/rrrTuu

where rrr = rack, T= brick type, and uu = location in rack.

Below is an example of hinv (IRIX hardware inventory command) output identifying the rack and bay numbers:

$ hinv 
/hw/module/001c10 
/hw/module/002r19

A rack is numbered with a three-digit number. A compute rack (designated for C–bricks) is numbered sequentially beginning with 001. An I/O rack (designated for I–bricks, P–bricks, X–bricks, and D–bricks) is numbered sequentially (also beginning with 001) and by the physical quadrant in which the I/O rack resides.

In a single-rack configuration (for all 3200 server configurations and some 3400 server configurations), the compute rack is numbered 001 even if it contains an I/O brick or D–brick.

Figure 1-6 shows the rack numbering scheme for multiple-rack server systems.

Figure 1-6. Rack Numbering

Rack numbers are represented as decimal numbers in the hardware graph path, for example:

$ hinv 
/hw/module/001C10

The SGI Origin 3000 server series use caches to reduce memory latency. Although data exists in local or remote memory, copies of the data can exist in various processor caches throughout the system. Cache coherency keeps the cached copies consistent.

To keep the copies consistent, the ccNUMA architecture uses directory-based coherence protocol. In directory-based coherence protocol, each block of memory (128 bytes) has an entry in a table that is referred to as a directory. Like the blocks of memory that they represent, the directories are distributed among the C–bricks. A block of memory is also referred to as a cache line.

Each directory entry indicates the state of the memory block that it represents. For example, when the block is not cached, it is in an unowned state. When only one processor has a copy of the memory block, it is in an exclusive state. And when more than one processor has a copy of the block, it is in a shared state; a bit vector indicates which caches contain a copy.

When a processor modifies a block of data, the processors that have the same block of data in their caches must be notified of the modification. The SGI Origin 3000 server series use an invalidation method to maintain cache coherence. The invalidation method purges all unmodified copies of the block of data and the processor that wants to modify the block receives exclusive ownership of the block.

In DSM systems, memory is physically located at various distances from the processors. As a result, memory access times (latencies) are different or “non–uniform.” For example, it takes less time for a processor to reference its local memory than it does to reference remote memory.

In a NUMA system, program performance is based on proper placement of important data structures. In general, data should be located close to the processor that will access it. IRIX provides a service to enable applications to achieve this.