In order to protect data integrity, nodes without system controllers must use the I/O fencing feature, which isolates a problem node so that it cannot access I/O devices and therefore cannot corrupt data in the shared CXFS filesystem.
This feature can only be used with a Brocade Fibre Channel switch supported by SGI; therefore, the Brocade switch is a required piece of hardware in a multiOS cluster.
The Brocade Fibre Channel switches will be initially installed and configured by SGI personnel. You can use the information in this chapter to verify the installation.
This release supports Brocade Silkworm Fibre Channel switches that are supported by SGI:
2400 (8-port)
2800 (16-port)
3200 (8-port, 2-Gbit)
3800 (16-port, 2-Gbit)
3900 (32-port, 2-Gbit)
12000 (32-, 64-, or dual 64-port, 2-Gbit)
All Brocade switches contained within the SAN fabric must have the appropriate Brocade license key installed. The following firmware is required:
2400 and 2800 switches: v2.6.1 or later
3200 and 3800 switches: v3.1.1 or later
3900 and 12000 switches: v4.1.1 or later
If the current firmware level of the switches must be upgraded, please contact your local SGI service representative or customer support center.
The Brocade switch must be configured so that its Ethernet interface is accessible from all administration nodes using telnet. The fencing network connected to the Brocade switch must be physically separate from the private heartbeat network.
 | Caution: The telnet port must be kept free in order for I/O fencing to succeed. |
The 3900 and 12000 series switches permit multiple telnet sessions. However, CXFS I/O fencing requires a telnet lockout for global mutual exclusion when a fencing race occurs. Therefore, you must configure the 3900 and 12000 series switches to set the maximum allowed simultaneous telnet sessions for the admin user to one. Only the 3900 and 12000 switches require this configuration (other Brocade switch models are shipped with the required restrictions configured by default). See “Configuring the Brocade Silkworm 3900”, and “Configuring the Brocade Silkworm 12000”.
To verify the Brocade license, log into the switch as user admin and use the licenseshow command, as shown in the following example:
brocade:admin> licenseshow
dcRyzyScSedSz0p:
Web license
Zoning license
SES license
Fabric license
SQQQSyddQ9TRRdUP:
Release v2.2 license |
To verify the firmware version, log into the switch as user admin and use the version command, as shown in the following example:
workstation% telnet brocade1 Trying 169.238.221.224... Connected to brocade1.acme.com Escape character is '^]'. Fabric OS (tm) Release v2.6.0d login: admin Password: brocade1:admin> version Kernel: 5.4 Fabric OS: v2.6.0d <== Firmware Revision Made on: Fri May 17 16:33:09 PDT 2002 Flash: Fri May 17 16:34:55 PDT 2002 BootProm: Thu Jun 17 15:20:39 PDT 1999 brocade1:admin> |
To limit the maximum allowed simultaneous telnet sessions for the admin user to one on the Brocade Silkworm 3900, do the following:
Connect to the switch via the telnet command and login as root.
Issue the sync command to avoid filesystem corruption:
# sync
Edit the /etc/profile file to change the max_telnet_sessions from 2 to 1 and place the information in a new file. For example:
# cd /etc # sed -e 's/max_telnet_sessions=2/max_telnet_sessions=1/' profile >profile.new
Distribute the edited profile file to both partitions on both central processors. For example:
# cp profile.new profile # cp profile.new /mnt/etc/profile
Issue the sync command again to avoid filesystem corruption:
# sync
To limit the maximum allowed simultaneous telnet sessions for the admin user to one on the Brocade Silkworm 12000, do the following:
Connect to the switch via the telnet command and login as root.
Use the haShow command to make sure that both central processors are up. This is indicated by the message Heartbeat Up within the output of the haShow command. If it is not up, wait a couple of minutes and run haShow again to check for the status.
Issue the sync command on the filesystems to avoid filesystem corruption:
# rsh 10.0.0.5 sync # rsh 10.0.0.6 sync
Edit the /etc/profile file to change the max_telnet_sessions from 2 to 1 and place the information in a new file. For example:
# cd /etc # sed -e 's/max_telnet_sessions=2/max_telnet_sessions=1/' profile >profile.new
Distribute the new profile to both partitions and central processors. For example:
# rcp /etc/profile.new 10.0.0.5:/etc/profile # rcp /etc/profile.new 10.0.0.5:/mnt/etc/profile # rcp /etc/profile.new 10.0.0.6:/etc/profile # rcp /etc/profile.new 10.0.0.6:/mnt/etc/profile
Issue the sync command again to avoid filesystem corruption:
# rsh 10.0.0.5 sync # rsh 10.0.0.6 sync
To change Brocade Fibre Channel cable connections used by nodes in the CXFS cluster, do the following:
Cleanly shut down CXFS services on the nodes affected by the cable change using either the CXFS GUI or the cmgr command.
Rearrange the cables as required.
Restart CXFS services.
Reconfigure I/O fencing if required. You must perform this step if I/O fencing is enabled on the cluster and if you added/removed any Brocade switches. You must use the CXFS GUI or the cmgr command to add/remove switches from the CXFS configuration as required.
If any CXFS client nodes are connected to a new (or different) Brocade switch, restart CXFS services on those nodes. This will ensure that the administration nodes can correctly identify the Brocade ports used by all clients.