Chapter 4. Administering GRIO

Chapter 4. Administering GRIO
Prev		Next

You can use the cxfs_admin(1M) or cmgr(1M) command to configure a qualified bandwidth and activate a filesystem as GRIO-managed. You can use the grioadmin tool to perform node-level administration tasks for local XFS and shared CXFS filesystems, such as the following:

List active GRIO reservations on the current node
Create, modify, and release node-level bandwidth allocations
Query available bandwidth

`grioadmin` Command Line

grioadmin [options] [fs|streamID]

The arguments are as follows:

`-a`	Prints the bandwidth available for reservation for the specified filesystem and the total bandwidth currently reserved for use by the local node, either allocated to the node or reserved by applications. This total includes bandwidth temporarily allocated to the node by the distributed bandwidth allocator (DBA) running within `ggd2`.
`-C`	Indicates that the node-level allocation requested with the `-g` or `-m` option should be at most bw MB/s (a ceiling). This is the default behavior for `-g`.
`-F`	Indicates that the node-level allocation requested with the `-g` or `-m` option should be at least bw MB/s (a floor).
`-g bw`	Allocates bw MB/s for access by the local node to the specified filesystem, if bandwidth is available. This node-level allocation is shared by all applications running on the node that are not GRIO aware. The allocation type can be either a ceiling (`-C`) or a floor (`-F`). The default allocation type is a ceiling .
`-h`	Print a usage message.
`-l`	Lists active streams in an easily parsed form (one per line).
`-L cell`	Relocates the GRIO server to the node with the cell ID cell. To determine the cell ID, use any of the standard CXFS reporting utilities, such as `cxfs_admin`, `cxfs_info`, or `clconf_info`.
`-m bw`	Modifies the current node-level allocation on the specified filesystem to a bandwidth of bw MB/s. You can modify the type of an active allocation by also specifying either `-C` or `-F`.
`-r`	Releases the current node-level allocation on the specified filesystem.
`-s`	Prints a more human-readable summary of active streams than `-l`. The results are grouped per filesystem.
`-v`	Displays verbose output (used with `-s`).
streamID	Specifies the ID of an active GRIO stream.
fs	Specifies a path that identifies a mounted GRIO-managed filesystem (in which case the non-GRIO stream for the filesystem is used).

If you specify grioadmin without any arguments, it prints a usage message by default.

`grioadmin` Examples

The following example commands create, inspect, modify, and remove a node-level allocation of the GRIO-managed filesystem /stripe with a qualified bandwidth of 350 MB/s.

Note: grioadmin classifies active streams as one of the followings:

App, which is a GRIO stream that is created explicitly by an application using the interfaces in the GRIO library. Only I/O to file descriptors that have been bound to such a stream using grio_bind(3X) will receive the requested QoS guarantee.
Dynamic, which is the component of the node-level allocation that is controlled by the DBA running within ggd2. It is updated periodically based on the amount of free bandwidth in the filesystem and the I/O demand from different nodes.
Static, which is the component of the node-level allocation that has been configured by an administrator using grioadmin with the -g option. The Static component is not affected by the operation of the DBA or other App reservations, and persists until you remove it manually by using the -r option. You can configure a node-level allocation as either a floor or a ceiling. The grioadmin -s option output indicates the type of active node-level allocations by printing a FLOOR or CEIL designator with the Static component of the currently allocated bandwidth.

Query the available bandwidth on the /stripe filesystem and show the active streams:
$ grioadmin -a /stripe 349.94 MB/s available on /stripe 1.00 MB/s allocated to this node $ grioadmin -s /stripe: Dynamic 1.00 MB/s
The output from these commands shows the following:
- The minimum dynamic allocation that ggd2 is configured to make to an idle node is 1.00 MB/s
- There are just under 350 MB/s available to be reserved
Create a 200-MB/s node-level allocation for this node in the /stripe filesystem:
$ grioadmin -g200 /stripe Static allocation configured for /stripe. Stream ID is d919c6e5-8405-1029-8d74-08006913a7f7
A stream will be created for the allocation and is attached to the non-GRIO stream in the kernel.

Verify that the available bandwidth was reduced accordingly:

$ grioadmin -a /stripe
149.94 MB/s available on /stripe
200.06 MB/s allocated to this node

Inspect the kernel streams:

$ grioadmin -s
/stripe:
  Dynamic         1.00 MB/s
  Static        200.00 MB/s  CEIL

This shows that the /stripe filesystem has its initial minimal Dynamic allocation and a new node-level (Static) allocation.

To be more verbose and print the stream IDs, add the -v option:

$ grioadmin -sv 
/stripe:
  Dynamic         1.00 MB/s  b77c9351-7b63-1029-8f56-08006913a7f7
  Static        200.00 MB/s  d919c6e5-8405-1029-8d74-08006913a7f7  CEIL

Increase the node-level allocation to 300 MB/s without disturbing in-progress I/O on the node:

$ grioadmin -m300 /stripe
Static bandwidth allocation for filesystem /stripe has
been modified:
$ grioadmin -s
/stripe:
  Dynamic         1.00 MB/s
  Static        300.00 MB/s  CEIL

Start a GRIO-aware application that requests a 50-MB/s reservation and print the stream IDs:
$ grioadmin -s /stripe /stripe: Dynamic 1.00 MB/s Static 200.00 MB/s CEIL App (31932) 50.00 MB/s
This output shows that the new App stream has a process ID of 31932.

Terminate the application and remove the node-level allocation:

$ grioadmin -r /stripe
Static bandwidth allocation for filesystem /stripe has
been released.
$ grioadmin -s
/stripe:
  Dynamic         1.00 MB/s

The following examples show using grioadmin on a Windows client to create a floor reservation (-F) , modify it to a ceiling allocation (-C) , and then release it:

C:\>grioadmin -sv
GRIO cluster server is cxfsaltix1 (cell 0)
x:\mnt\tp9500_1: cluster:
  Dynamic         1.00 MB/s  6016cd36-8900-2a10-b682-9c12fb0816e8

C:\>grioadmin -F -g 10 x:\mnt\tp9500_1
Static allocation configured for x:\mnt\tp9500_1.
Stream ID is a267e371-caca-324f-3b89-f8c41eaca4b0.

C:\>grioadmin -sv
GRIO cluster server is cxfsaltix1 (cell 0)
x:\mnt\tp9500_1: cluster:
  Dynamic         1.00 MB/s  6016cd36-8900-2a10-b682-9c12fb0816e8
  Static         10.00 MB/s  a267e371-caca-324f-3b89-f8c41eaca4b0 FLOOR

C:\>grioadmin -C -m 10 x:\mnt\tp9500_1
Static bandwidth allocation for filesystem x:\mnt\tp9500_1 has been modified.

C:\>grioadmin -sv
GRIO cluster server is cxfsaltix1 (cell 0)
x:\mnt\tp9500_1: cluster:
  Dynamic         1.00 MB/s  6016cd36-8900-2a10-b682-9c12fb0816e8
  Static         10.00 MB/s  a267e371-caca-324f-3b89-f8c41eaca4b0 CEIL

C:\>grioadmin -r x:\mnt\tp9500_1
Static bandwidth allocation for filesystem x:\mnt\tp9500_1 has been released.

C:\>grioadmin -sv
GRIO cluster server is cxfsaltix1 (cell 0)
x:\mnt\tp9500_1: cluster:
  Dynamic         1.00 MB/s  6016cd36-8900-2a10-b682-9c12fb0816e8

Figure 4-1 through Figure 4-3 show cxfs_info examples on a Windows node.

Figure 4-1. Ceiling (Max) Node-Level Allocation

Figure 4-2. Floor (Min) Node-Level Allocation

Figure 4-3. No Node-Level Allocation

Prev	Table of Contents	Next
Chapter 3. Setting Up GRIO		Chapter 5. Monitoring Quality of Service