The rackmount Onyx2 provides features such as

Other features of the Onyx2 rackmount system include

The single-rack Onyx2 is illustrated in Figure 1-1. The rack holds one graphics module and one processor compute module.

Figure 1-2 shows a block diagram example of the graphics rack's potential processor and graphics pipe configurations. The example shows a configuration with four Node boards and two fully loaded graphics pipes. Each system will vary based on the configuration ordered.

The multirack Onyx2 systems come in several configurations. The multirack Onyx2 system illustrated in Figure 1-3 is a dual rack system. This system can hold up to four modules. The example diagrams in Figure 1-4 and Figure 1-5 show a multirack system with a single compute module and three or four graphics pipes. The multirack Onyx2 system illustrated in Figure 1-6 is a triple rack system, also known as a RealityMonster system. From a software perspective, “monster mode” is used when more than one graphics pipe renders an image to a single video channel.

The configuration shown in Figure 1-6 can hold up to six modules (two compute modules and four graphics modules) and is available in different configurations. The example diagrams in Figure 1-7 and Figure 1-8 show a RealityMonster configuration of sixteen processors and eight graphics pipes. A RealityMonster configuration always requires an Ethernet hub to connect the multimodule System Controllers (MMSCs).

Figure 1-9 shows a four-rack configuration with six pipes and 12 processors. Each pipe uses four RM/TM assemblies.

The Onyx2 graphics subsystem contains one or two InfiniteReality2 board sets (pipes). One pipe holds one, two, or four RM/TM assemblies and the second uses one or two RM/TM assemblies. The major components of the graphics subsystem are

The major hardware components include the

Each of these components are standard in the computing module.

The standard I/O subsystem consists of a BaseIO board assembly (also known as IO6G) that supports:

Additional I/O connection capabilities are available with optional XIO boards.

The Node boards within the Onyx2 compute module use links that differ from bus technology. While a bus is a resource that can be used by only one processor at a time, the communications “fabric” in the Onyx2 rack makes connections from processor to processor as they are needed. Each Node board contains either one or two processors, a portion of main memory, a directory to maintain cache coherence, and two interfaces:

This web of connections differs from a bus in the same way that multiple dimensions differ from a single dimension. You could describe a bus as a one-dimensional line while the Onyx2 uses a multidimensional mesh.

The multiple data paths used are constructed as they are needed by router ASICs, which act as switches. When you add a Node board, you add to and scale the system bandwidth.

In the case of a multirack graphics system with more than one compute module, the CrayLink Interconnect links modules to one another. The CrayLink Interconnect may appear to be a type of super data bus, but it differs from a bus in several important ways. Basically, a bus is a resource that can be used only by one processor at a time. The Craylink Interconnect is a mesh of multiple, simultaneous, dynamically allocable connections that are made from processor to processor in the connected compute modules.

This makes the Onyx2 multirack system very scalable because it can range in size from 4 to 64 processors. As you add Node boards or compute modules, you add to and scale the system bandwidth.

The CrayLink Interconnect uses a set of switches, called routers, that are linked by cables in various configurations, or topologies. Here are some key features:

The CrayLink Interconnect provides a minimum of two separate paths to any pair of compute modules. This redundancy allows the system to bypass failing routers or broken fabric links. Each fabric link is additionally protected by a CRC code and a link-level protocol, which retry any corrupted transmissions and provide fault tolerance for transient errors.

Main memory on each Node board in the system can be distributed and shared among the system microprocessors. This shared memory is accessible to all processors in a compute module through the S2MP architecture. The CrayLink interconnection fabric provides inter-module accesses with low latency.

Each Node board in the graphics rack system is an independent memory source, and each Node board is capable of optionally supporting up to 4 GB of memory. A directory memory keeps track of information necessary for hardware coherency and protection.

Each Node board uses a HUB ASIC that is the distributed shared-memory controller. This ASIC is responsible for providing all of the processors and I/O devices with transparent access to all of distributed memory in a cache-coherent manner. Cache coherence is the ability to keep data consistent throughout a system. In the Onyx2 system, data can be copied and shared amongst all the processors and their caches. Moving data into a cache may cause the cached copy to become inconsistent with the same data stored elsewhere. The cache coherence protocol is designed to keep data consistent and to disperse the most-recent version of data to wherever it is being used.

Although memory is physically dispersed across the system Node boards, special page migration hardware moves data into memory closer to a processor that frequently uses it. This page migration scheme reduces memory latency—the time it takes to retrieve data from memory. Although main memory is distributed, it is universally accessible and shared between all the processors in the system. Similarly, I/O devices are distributed among the Nodes, and each device is accessible to every processor in the system.

The Onyx2 graphics rack's microprocessor “brains” and primary memory are located on a processor board called a Node board. Each Node board in the Onyx2 graphics rack can house one or two 64-bit microprocessors. Each microprocessor uses a customized two-way interleaved data cache, and has dedicated second-level cache support. A high-performance bus interface links the CPU directly with supporting SRAM.

The Node board's main memory slots can be populated with 32 MB or 64 MB memory modules. See Figure 1-10 for an example Node board illustration.

Note that directory memory is used only in large scale multirack systems; there is no reason to use directory memory in a single-rack system.

XIO boards give the graphics rack system a wide range of optional interfaces in a manner similar to VME interfaces. Optional XIO boards can support communication interfaces such as

Check with your SGI sales or support representative for information on these or other optional interfaces available on XIO boards.

Each processor compute module in an Onyx2 graphics rack system uses a midplane. This means that boards, disk drives, and other devices can plug into both sides of the system. This allows for maximum functionality and expansion.

Single-ended Ultra SCSI and SCSI-2 disk, CD-ROM, and tape drives are the only devices internally supported by the Onyx2 graphics rack systems.

Figure 1-11 shows the front of the processor compute module's midplane.

The rack system's graphics module uses a midplane in a similar way to the processor compute module. However, it does not have any disks or CD-ROM connections on the front. Only the MSC and power supply connections are made in the front. All graphics and interface boards go in the rear of the module and connect to the back of the midplane.

Figure 1-12 shows a view of the graphics module midplane.