Chapter 4. GRIO API Overview

User processes communicate with the ggd2 daemon using the following core library calls:

The process that initially reserves bandwidth by calling grio_reserve() or grio_reserve_fd() is referred to as the owning process. Any streams not already released when their owning process exits will be automatically released. Processes can share streams. The ownership of a GRIO stream is nontransferable.

grio_avail()

Synopsis:

#include <grio2.h>

int grio_avail(
         const char *fs,
         grio_off_t *bytes, grio_msecs_t *msecs)

cc ... -lgrio2

The grio_avail() call returns the currently available guaranteed-rate bandwidth for a specified filesystem. The returned bandwidth is the qualified bandwidth of the filesystem minus bandwidth reserved for active GRIO streams and any bandwidth statically allocated for nonguaranteed-rate I/O. While ggd2 temporarily allows unreserved bandwidth to be used for servicing nonguaranteed-rate I/O, this bandwidth is reclaimed when a GRIO reservation is received and is therefore considered available.

For more information, see the grio_avail (3X) man page.

grio_bind()

Synopsis:

#include <grio2.h>
          
int grio_bind(grio_descriptor_t fd, grio_stream_id_t *stream_id);

          cc ... -lgrio2

The grio_bind() call binds one or more open file descriptors to a GRIO stream. Once bound, all I/O to or from the file descriptors will receive the quality-of-service guarantees of the stream.

Binding a file descriptor increments the reference count of a stream by 1. The file descriptor remains bound to the stream until it is either closed or explicitly unbound with grio_unbind().

The file descriptor must be capable of GRIO I/O. That is, it must refer to an open file on an XFS or CXFS filesystem and be configured for direct I/O.

For more information, see the grio_bind (3X) man page.

grio_get_stream()

Synopsis:

#include <grio2.h>

int grio_get_stream(
          grio_descriptor_t fd,
          grio_stream_id_t *stream_id);

cc ... -lgrio2

The grio_get_stream() call returns the ID of the stream to which it is bound.

For more information, see the grio_get_stream (3X) man page.

grio_modify()

Synopsis:

#include <grio2.h>

int grio_modify(
          grio_stream_id_t *stream_id,
          grio_off_t *bytes, grio_msecs_t *msecs,
          int flags)

cc ... -lgrio2

The grio_modify() call changes the properties of an existing GRIO stream. You can increase or decrease reserved bandwidth by using this call.


Note: grio_modify() is a synchronous call and, when increasing a reservation, may block while bandwidth is reallocated. This delay can be in the order of 1 or 2 seconds and applications should be designed to accommodate this delay if necessary. While a call to grio_modify() is being processed, I/O to the stream continues uninterrupted at its existing rate.

For more information, see the grio_modify (3X) man page.

grio_release()

Synopsis:

#include <grio2.h>

int grio_release(grio_stream_id_t *stream_id)

cc ... -lgrio2

The grio_release() call removes a GRIO stream ID from the system and releases the primary reference taken when it was created. When all remaining references to the stream are removed, the stream will be destroyed and its associated bandwidth will be returned to the system.

The grio_release() call hides the stream. Attempts to bind new file descriptors using grio_bind() will fail with a return value of ENOENT. However, the quality-of-service guarantees of the stream will remain in effect until all remaining bound file descriptors are either unbound or closed, and any in-flight I/O to the stream completes.

This behavior gives an application some flexibility in controlling the extent of a GRIO guarantee. For instance, by binding a file descriptor to a stream and immediately releasing the stream, an application can create a temporary reservation that persists for as long as the file descriptor remains open. Alternatively, if a process does not explicitly release a stream, the guarantee persists until that process exits.

For more information, see the grio_release (3X) man page.

grio_reserve() and grio_reserve_fd()

Synopsis:

#include <grio2.h>

int grio_reserve(
          const char *path,
          grio_off_t *bytes, grio_msecs_t *msecs,
          int flags,
          grio_stream_id_t *stream_id)

int grio_reserve_fd(
          grio_descriptor_t fd,
          grio_off_t *bytes, grio_msecs_t *msecs,
          int flags,
         grio_stream_id_t *stream_id)

cc ... -lgrio2

The grio_reserve() and grio_reserve_fd() calls reserve guaranteed rate bandwidth to or from a GRIO-managed filesystem. If successful, they set up a GRIO stream in the kernel with the requested properties and return its stream ID:

  • grio_reserve() makes a filesystem-level reservation. The target filesystem is identified with a path that must be either the filesystem mount point or the device special file on which it is located. Before guaranteed rate I/O can be performed, an open file descriptor must be bound to the new stream using grio_bind().

  • grio_reserve_fd() makes a file-level reservation. It takes an open file descriptor on the target filesystem. In addition to reserving bandwidth, the file descriptor is bound to the newly created stream. The file must therefore be suitable for guaranteed rate I/O and satisfy the normal requirements of grio_bind.

The requested bandwidth is specified as the number of bytes delivered every msecs milliseconds. msecs is referred to as the reservation interval . This value provides additional information to the GRIO scheduler about an application's ability to tolerate variation in I/O service time. (For example, an application request of 1MB every tenth of a second suggests a tighter requirement than 100 MB delivered every second, even though both requests describe the same average data rate.) GRIO uses this information as a hint only, and honors the expressed bandwidth over an implementation-defined scheduling interval.

grio_release() should be called when the stream is no longer required.

The process that creates a stream with these calls is said to be the owning process. Any streams not already released when their owning process exits will be automatically released. The ownership of a GRIO stream is not transferable.

GRIO streams are reference-counted. When created, a new stream has a reference count of 1. This primary reference remains until the reservation is released using grio_release() or the owning process exits.

A stream persists until its reference count drops to 0. Binding a file descriptor using grio_bind() adds a reference. Unbinding using grio_unbind() or closing a file descriptor removes a reference. In-flight I/O will also add references to a stream for short periods of time.

It is possible, and frequently useful, for a stream to persist after it has been released. For more information, see the grio_release() man page.


Note: Both grio_reserve() and grio_reserve_fd() are synchronous calls and may block while bandwidth is reallocated. This delay is referred to as the stream creation latency .

In the worst case, this delay can be in the order of 1 or 2 seconds, although it may be significantly less depending on the configuration of a particular GRIO deployment.

The following are strategies to minimize the impact of this behavior:

  • Reserve bandwidth well ahead of time

  • Perform reservations in a dedicated thread

  • Reuse a reservation wherever possible

  • Configure ggd2 to keep a proportion of the available free bandwidth uncommitted using the -r option.


For more information, see the grio_reserve (3X) and ggd2(1M) man pages.

grio_unbind()

Synopsis:

#include <grio2.h>

int grio_unbind(grio_descriptor_t fd);

cc ... -lgrio2

The grio_unbind() call unbinds a file descriptor from its GRIO stream. Unbinding a file descriptor decrements the reference count of its stream by 1.

Once unbound, I/O to or from the file descriptor may continue, but will be scheduled as regular, non-guaranteed rate I/O.

For more information, see the grio_unbind(3X) man page.