Appendix B. XDR Protocol Specification

This chapter describes the XDR protocol, a standard for describing and encoding data. The XDR standard assumes that bytes (or octets) are portable, where a byte is defined as 8 bits of data. It also assumes that hardware that encodes bytes onto various media will preserve the bytes' meanings across hardware boundaries. (For example, the Ethernet standard suggests that bytes be encoded in “little-endian” style, or least significant bit first.)

Once XDR data is shared among machines, it shouldn't matter that the data produced on an IRIS is consumed by a VAX (or vice versa). Similarly, the choice of operating systems should have no influence on how the data is represented externally. For programming languages, data produced by a C program should be readable by a FORTRAN or Pascal program.

Topics in this chapter include:

Basic Block Size

Representation of all items requires a multiple of four bytes (32 bits) of data. The bytes are numbered 0 through n–1. The bytes are read or written to some byte stream such that byte m always precedes byte m+1. If the number of bytes needed to contain the data are not a multiple of 4, those n bytes are followed by enough (0 to 3) residual zero bytes, r, to make the total byte count a multiple of 4.

Include the familiar graphic box notation for illustration and comparison. In most illustrations, each box (delimited by a plus sign at each of the four corners, and vertical bars and dashes) depicts a byte. Ellipses (...) between boxes indicate zero or more additional bytes, where required.

Block

+--------+--------+...+--------+--------+...+--------+
| byte 0 | byte 1 |...|byte n-1|    0   |...|    0   |
+--------+--------+...+--------+--------+...+--------+
|<-----------n bytes---------->|<------r bytes------>|
|<-----------n+r (where (n+r) mod 4 = 0)>----------->|

XDR Data Types

This section describes the data types defined in the XDR standard, showing how each data type is declared in XDR language, and including a graphic representation of how each type is encoded.

Integers

An XDR signed integer is a 32-bit datum that encodes an integer in the range [–2147483648, 2147483647]. The integer is represented in two's complement notation. The most and least significant bytes are 0 and 3, respectively. The data description of integers is integer.

Integer

(MSB)                   (LSB)
+-------+-------+-------+-------+
|byte 0 |byte 1 |byte 2 |byte 3 |
+-------+-------+-------+-------+
<------------32 bits------------>

Unsigned Integers

An XDR unsigned integer is a 32-bit datum that encodes a non-negative integer in the range [0,4294967295]. It is represented by an unsigned binary number whose most and least significant bytes are 0 and 3, respectively. The data description of unsigned integers is unsigned.

Unsigned Integer

(MSB)                   (LSB)
+-------+-------+-------+-------+
|byte 0 |byte 1 |byte 2 |byte 3 |
+-------+-------+-------+-------+
<------------32 bits------------>

Enumerations

Enumerations have the same representation as integers and are handy for describing subsets of integers. The data description of enumerated data is:

enum { name-identifier = constant, ... } identifier;

The three colors (red, yellow, and blue) could be described by an enumerated type, as follows:

enum { RED = 2, YELLOW = 3, BLUE = 5 } colors;

It is an error to encode enumerations that have not been given assignments in the enum declaration.

Booleans

Booleans are important enough and occur frequently enough to warrant their own explicit type in the standard.

Booleans are declared like this:

bool identifier; 

This is equivalent to:

enum { FALSE = 0, TRUE = 1 } identifier; 

Hyper Integers and Hyper Unsigned

The XDR standard also defines 64-bit (8-byte) numbers called hyper integers and unsigned hyper integers. Their representations are the obvious extensions of the integer and unsigned integer defined in the preceding sections in this chapter. They are represented in two's complement notation. The most and least significant bytes are 0 and 7, respectively.

Hyper Integer or Unsigned Hyper Integer

(MSB)                                            (LSB)
+------+------+------+------+------+------+------+------+
|byte 0|byte 1|byte 2|byte 3|byte 4|byte 5|byte 6|byte 7|
+------+------+------+------+------+------+------+------+
<-------------------------64 bits----------------------->

Floating Points

The XDR standard defines the floating-point data type float (32 bits or 4 bytes). The encoding used is the IEEE standard for normalized single–precision floating-point numbers. (See the ANSI/IEEE 754-1985 floating-point standard.)

The following fields describe the single-precision floating-point number:

S 

The sign of the number. Values 0 and 1 represent positive and negative, respectively. One bit.

E 

The exponent of the number, base 2. Eight bits are devoted to this field. The exponent is biased by 127.

F 

The fractional part of the number's mantissa, base 2. Twenty-three bits are devoted to this field.

Therefore, the floating-point number is described by:

(-1)S * 2{E - Bias} * 1.F

It is declared like this:

Single-Precision Floating-Point

+-------+-------+-------+-------+
|byte 0 |byte 1 |byte 2 |byte 3 |
S|   E   |           F          |
+-------+-------+-------+-------+
1|<- 8 ->|<-------23 bits------>|
<------------32 bits------------>

Just as the most and least significant bytes of a number are 0 and 3, the most and least significant bits of a single-precision, floating-point number are 0 and 31. The beginning bit (and most significant bit) offsets of S, E, and F are 0, 1, and 9, respectively. Note that these numbers refer to the mathematical positions of the bits, and not to their actual physical locations (which vary from medium to medium).

The IEEE specifications should be consulted regarding the encoding for signed zero, signed infinity (overflow), and denormalized numbers (underflow). According to IEEE specifications, the “NaN” (not a number) is system dependent and should not be used externally.

Double-Precision Floating Points

The XDR standard defines the encoding for the double-precision floating–point data type double (64 bits or 8 bytes). The encoding used is the ANSI/IEEE 754/1985 standard for normalized double-precision, floating–point numbers.

The following fields describe the double-precision floating-point number:

S 

The sign of the number. Values 0 and 1 represent positive and negative, respectively. One bit.

E 

The exponent of the number, base 2. Eleven bits are devoted to this field. The exponent is biased by 1023.

F 

The fractional part of the number's mantissa, base 2. Fifty-two bits are devoted to this field.

It is declared as follows:

Double-Precision Floating-Point

+------+------+------+------+------+------+------+------+
|byte 0|byte 1|byte 2|byte 3|byte 4|byte 5|byte 6|byte 7|
S|    E   |                    F                        |
+------+------+------+------+------+------+------+------+
1|<--11-->|<-----------------52 bits------------------->|
<-----------------------64 bits------------------------->

Just as the most and least significant bytes of a number are 0 and 3, the most and least significant bits of a double-precision, floating-point number are 0 and 63. The beginning bit (and most significant bit) offsets of S, E, and F are 0, 1, and 12, respectively. Note that these numbers refer to the mathematical positions of the bits, and not to their actual physical locations (which vary from medium to medium).

The IEEE specifications should be consulted concerning the encoding for signed zero, signed infinity (overflow), and denormalized numbers (underflow). According to IEEE specifications, the “NaN” (not a number) is system dependent and should not be used externally.

Fixed-Length Opaque Data

At times, fixed-sized, uninterpreted data needs to be passed among machines. This data is called opaque and is declared like this:

opaque identifier[n];

Constant n is the (static) number of bytes necessary to contain the opaque data. If n is not a multiple of 4, the n bytes are followed by enough (0 to 3) residual zero bytes, r, to make the total byte count of the opaque object a multiple of 4.

Fixed-Length Opaque

0        1         ...
+--------+--------+...+--------+--------+...+--------+
| byte 0 | byte 1 |...|byte n-1|    0   |...|    0   |
+--------+--------+...+--------+--------+...+--------+
|<-----------n bytes---------->|<------r bytes------>|
|<-----------n+r (where (n+r) mod 4 = 0)------------>|

Variable-Length Opaque Data

The XDR standard also provides for variable-length (counted) opaque data, defined as a sequence of n (numbered 0 through n–1) arbitrary bytes to be the number n encoded as an unsigned integer and followed by the n bytes of the sequence.

Byte m of the sequence always precedes byte m+1 of the sequence, and byte 0 of the sequence always follows the sequence's length (count). Enough residual zero bytes (0 to 3), r, make the total byte count a multiple of 4.

Variable-length opaque data is declared like this:

opaque identifier<m>; 

or

opaque identifier<>; 

The constant m denotes an upper bound of the number of bytes the sequence may contain. If m is not specified, as in the second declaration, it is assumed to be the maximum length, (232) –1.

The constant m would normally be found in a protocol specification. For example, a filing protocol may state that the maximum data transfer size is 8192 bytes, as follows:

opaque filedata<8192>;

It is an error to encode a length greater than the maximum described in the specification.

Variable-Length Opaque

0     1     2     3     4     5      ..
+-----+-----+-----+-----+-----+-----+..+-----+-----+..+-----+
|        length n       |byte0|byte1|..| n-1 |  0  |..|  0  |
+-----+-----+-----+-----+-----+-----+..+-----+-----+..+-----+
|<-------4 bytes------->|<------n bytes----->|<---r bytes-->|
                        |<---n+r (where (n+r) mod 4 = 0)--->|

Strings

The XDR standard defines a string of n (numbered 0 through n–1) ASCII bytes to be the number n encoded as an unsigned integer, and followed by the n bytes of the string. Byte m of the string always precedes byte m+1 of the string, and byte 0 of the string always follows the string's length. If n is not a multiple of 4, the n bytes are followed by enough (0 to 3) residual zero bytes, r, to make the total byte count a multiple of 4.

Counted byte strings are declared as follows:

string object<m>; 

or

string object<>; 

The constant m denotes an upper bound of the number of bytes that a string may contain. If m is not specified, as in the second declaration, it is assumed to be the maximum length, (232) –1.

The constant m would normally be found in a protocol specification. For example, a filing protocol may state that a filename can be no longer than 255 bytes, as follows:

string filename<255>;

It is an error to encode a length greater than the maximum described in the specification.

String

0     1     2     3     4     5      ..
+-----+-----+-----+-----+-----+-----+..+-----+-----+..+-----+
|        length n       |byte0|byte1|..| n-1 |  0  |..|  0  |
+-----+-----+-----+-----+-----+-----+..+-----+-----+..+-----+
|<-------4 bytes------->|<-----n bytes------>|<---r bytes-->|
                        |<---n+r (where (n+r) mod 4 = 0)--->|

Fixed-Length Arrays

Declarations for fixed-length arrays of homogeneous elements are in this form:

type-name identifier[n];

Fixed-length arrays of elements numbered 0 through n–1 are encoded by individually encoding the elements of the array in their natural order, 0 through n–1. Each element's size is a multiple of 4 bytes. Although all elements are of the same type, the elements may have different sizes. For example, in a fixed-length array of strings, all elements are type string, yet each element will vary in length.

Fixed-Length Array


+---+---+---+---+---+---+---+---+...+---+---+---+---+
|   element 0   |   element 1   |...|  element n-1  |
+---+---+---+---+---+---+---+---+...+---+---+---+---+
|<--------------------n elements------------------->|

Variable-Length Arrays

Counted arrays provide the ability to encode variable-length arrays of homogeneous elements. The array is encoded as the element count n (an unsigned integer) followed by the encoding of each of the array's elements, starting with element 0 and progressing through element n–1.

Declaration for variable-length arrays follow this form:

type-name identifier<m>; 

or

type-name identifier<>; 

The constant m specifies the maximum acceptable element count of an array; if m is not specified, as in the second declaration, it is assumed to be (232) –1.

It is an error to encode a value of n that is greater than the maximum described in the specification.

Counted Array


0  1  2  3 
+--+--+--+--+--+--+--+--+--+--+--+--+...+--+--+--+--+ 
|     n     | element 0 | element 1 |...|element n-1| 
+--+--+--+--+--+--+--+--+--+--+--+--+...+--+--+--+--+ 
|<-4 bytes->|<--------------n elements------------->| 

Structures

The data description for structures in the XDR standard is very similar to data description in standard C:

struct {
    component-declaration-A;
    component-declaration-B;
    ...  
} identifier; 

The components of the structure are encoded in the order of their declaration in the structure. Each component's size is a multiple of 4 bytes, although the components may be different sizes.

Discriminated Unions

A discriminated union is a type composed of a discriminant followed by a type selected from a set of prearranged types according to the value of the discriminant. The type of discriminant is either int, unsigned int, or an enumerated type, such as bool. The component types are called “arms” of the union, and are preceded by the value of the discriminant, which implies their encoding.

Discriminated unions are declared like this:

union switch (discriminant-declaration) {
    case discriminant-value-A:
    arm-declaration-A;
    case discriminant-value-B:
    arm-declaration-B;
    ...
    default: default-declaration; 
} identifier; 

Each case keyword is followed by a legal value of the discriminant. The default arm is optional; if not specified, a valid encoding of the union cannot take on unspecified discriminant values. The size of the implied arm is always a multiple of 4 bytes. The discriminated union is encoded as its discriminant followed by the encoding of the implied arm.

Discriminated Union


0   1   2   3 
+---+---+---+---+---+---+---+---+ 
|  discriminant |  implied arm  | 
+---+---+---+---+---+---+---+---+ 
|<---4 bytes--->| 

Voids

An XDR void is a 0-byte quantity. Voids are useful for describing operations that take no data as input or no data as output, and in unions, where some arms may contain data and others do not.

The declaration is:

void;

Void

  ++
  ||
  ++ 
--><-- 0 bytes

Constants

The data declaration for a constant follows this form:

const name-identifier = n; 

const defines a symbolic name for a constant; it does not declare any data. The symbolic constant may be used anywhere a regular constant can be used. For example, the following defines the symbolic constant DOZEN, equal to 12:

const DOZEN = 12; 

Typedefs

A typedef does not declare data either, but it serves to define new identifiers for declaring data. The syntax is:

typedef declaration; 

The new type name is actually the variable name in the declaration part of the typedef. For example, the following defines a new type called eggbox using an existing type called egg:

typedef egg eggbox[DOZEN]; 

Variables declared using the new type name have the same type as the new type name would have in the typedef, if it was considered a variable. For example, the following two declarations are equivalent to declaring the variable fresheggs:

eggbox    fresheggs; 
egg    fresheggs[DOZEN]; 

When a typedef involves a struct, enum, or union definition, there is another (preferred) syntax that may be used to define the same type.

In general, a typedef of this form can be converted to the alternative form by removing the typedef part and placing the identifier after the struct, union, or enum keyword, instead of at the end:

typedef <<struct, union, or enum definition>> identifier; 

For example, there are two ways to define the type bool:

typedef enum {    /* using typedef */
    FALSE = 0,
    TRUE = 1 
} bool; 

enum bool {       /* preferred alternative */
    FALSE = 0,
    TRUE = 1 
}; 

The reason this syntax is preferred is that you do not have to wait until the end of a declaration to figure out the name of the new type.

Optional Data

Optional data is a kind of union that occurs so frequently that it is given a special declaration syntax of its own:

type-name *identifier; 

This syntax is equivalent to the following union declaration:

union switch (bool opted) {
    case TRUE:
        type-name element;
    case FALSE:
        void; 
} identifier; 

This syntax is also equivalent to the following variable-length array declaration, since the boolean opted can be interpreted as the length of the array:

type-name identifier<1>; 

Optional data is useful for describing recursive data structures, such as linked lists and trees. The following example defines a type stringlist, which encodes lists of arbitrary length strings:

struct *stringlist {
    string item<>;
    stringlist next; 
}; 

It could be declared equivalently as a union:

union stringlist switch (bool opted) {
    case TRUE:
        struct {
            string item<>;
            stringlist next;
        } element;
    case FALSE:
        void; 
}; 

Or, it could be declared as a variable-length array:

struct stringlist<1> {
    string item<>;
    stringlist next; 
}; 

Both declarations obscure the intention of the stringlist type, however, so the optional-data declaration is preferred over both of them.

The optional-data type is also closely correlated with how recursive data structures are represented in high-level languages, such as Pascal and C. In these cases, recursive data structures use pointers. In fact, the syntax is the same as that of the C language for pointers.

Areas for Future Enhancement

The XDR standard lacks representations for bit fields and bitmaps, since the standard is based on bytes. Also missing are packed (or binary-coded) decimals.

It is not the intent of the XDR standard to describe every kind of data that people have ever sent (or will ever want to send) from machine to machine. It only describes the most commonly used data types of high-level languages, such as Pascal and C so applications written in these languages will be able to communicate easily over some medium.

One could imagine extensions to XDR that would let it describe almost any existing protocol, such as TCP. The minimum requirement for these extensions is support for different block sizes and byte orders. The XDR discussed here could then be considered the 4-byte, big-endian member of a larger XDR family.

Common Questions about XDR

This section attempts to answer questions you may have about XDR.

  • Why have a language for describing data?

    There are many advantages to using a data description language, such as XDR, over using diagrams. Languages are more formal than diagrams and lead to less ambiguous descriptions of data. Languages are also easier to understand and allow one to think of other issues instead of the low-level details of bit encoding. Also, there is a close analogy between the types of XDR and a high-level language such as C or Pascal, which makes the implementation of XDR encoding and decoding modules an easier task. Finally, the language specification itself is an ASCII string that can be passed from machine to machine to perform on-the-fly data interpretation.

  • Why is there only one byte order for an XDR unit?

    Supporting two byte orderings requires a higher level protocol for determining in which byte order the data is encoded. Since XDR is not a protocol, it cannot support two byte orderings. The advantage, however, is that data in XDR format can be written to a magnetic tape, for example, and any machine will be able to interpret it, since no higher level protocol is necessary for determining the byte order.

  • Why does XDR use big-endian byte order?

    Yes, it is unfair that XDR uses big-endian byte order, but having only one byte order means you have to be unfair to somebody. Many architectures, such as the MIPS R2000/3000, Motorola 68000, and IBM 370, support the big-endian byte order.

  • Why is the XDR unit four bytes wide?

    There is a trade-off in choosing the XDR unit size. Choosing a small size such as two makes the encoded data small but causes alignment problems for machines that aren't aligned on these boundaries. A large size such as eight means the data will be aligned on virtually every machine but causes the encoded data to grow too big. Four was chosen as a compromise. The four-byte data unit is big enough to support most architectures efficiently, except for rare machines such as the 8-byte-aligned Cray. Four is also small enough to keep the encoded data restricted to a reasonable size.

  • Why must variable-length data be padded with zeros?

    Forcing the padded bytes to be zero ensures that the same data is encoded into the same thing on all machines, enabling the encoded data to be meaningfully compared or checksummed.

  • Why is there no explicit data typing?

    Data typing has a relatively high cost for what small advantages it may have. One cost is the expansion of data due to the inserted type fields. Another is the added cost of interpreting these type fields and acting accordingly. Most protocols already know what type they expect, so data typing supplies only redundant information. However, one can still get the benefits of data typing using XDR. One way is to encode two things: first a string, which is the XDR data description of the encoded data, and then the encoded data itself. Another way is to assign a value to all the types in XDR, and then define a universal type that takes this value as its discriminant and for each value describes the corresponding data type.