The storage bit map file controls the available space on a volume; this file is listed in the master file directory as BITMAP.SYS;1. It contains a storage control block, which consists of summary information intended to optimize the Files–11 space allocation, and the bit map itself, which lists the availability of individual blocks.

The bad block file, which is listed in the master file directory as BADBLK.SYS;1, contains all the bad blocks on the volume. The system detects bad disk blocks dynamically and prevents their reuse once the files to which they are allocated have been deleted.

When the Backup utility (BACKUP) creates sequential disk save sets, it stores the save set file in the MFD.

For an explanation of user file directories and file specifications, see the VSI OpenVMS User's Manual.

The core image file is listed in the MFD as CORIMG.SYS;1. It is not supported by the operating system.

The volume set list file is listed in the MFD as VOLSET.SYS;1. This file is used only on relative volume 1 of a volume set. The file contains a list of the labels of all the volumes in the set and the name of the volume set.

The continuation file is listed in the MFD as CONTIN.SYS;1. This file is used as the extension file identifier when a file crosses from one volume to another volume of a loosely coupled volume set. This file is used for all but the first volume of a sequential disk save set.

The backup log file is listed in the MFD as BACKUP.SYS;1. This file is reserved for future use.

The pending bad block log file is listed in the MFD as BADLOG.SYS;1. This file contains a list of suspected bad blocks on the volume that are not listed in the bad block file.

This file contains the volume security profile and is managed with the SET/SHOW security commands.

CD–ROM media is divided into logical sectors that are assigned a unique logical sector number. Logical sectors are the smallest addressable units of a CD–ROM. Each logical sector consists of one or more consecutively ascending physical sectors as defined by the relevant recording standard. ?Logical sectors are numbered in ascending order. The value 0 is assigned to the logical sector with the lowest physical address containing recorded data. Each logical sector includes a data field made up of at least 2048 bytes—but, in all cases, the number must be a power of 2.

ISO 9660-formatted CD–ROM volumes include a system area and a data area. The reserved system area includes logical sectors 0 through 15. The data area includes the remaining logical sectors and is called volume space. Volume space is organized into logical blocks that are numbered in consecutively ascending logical block number order.

Logical blocks are made up of at least 512 bytes—but, in all cases, the number of bytes must be a power of 2. However, no logical block can be larger than a logical sector.

The data area may include one or more Volume Descriptors, File Descriptors, Directory Descriptors, and Path Tables. These entities collectively describe the volume and file structure of an ISO 9660-formatted CD–ROM. The Ancillary Control Process (ACP) that manages I/O access to the CD–ROM views the volume and file structure as an integral part of the base OpenVMS file system.

Logical Blocks Greater Than 512 Bytes

OpenVMS device drivers are designed to handle files made up of 512-byte blocks that are uniquely addressable. The ISO 9660 standard supports logical blocks that are greater than 512-bytes. The Files-11 C/D ACP solves this incompatibility by converting ISO 9660 logical-block-size requests into OpenVMS-block-size requests at the file system level.

Partial Data Blocks

Any logical block in an ISO 9660 file extent may be partially filled with data. RMS assumes that all file blocks are filled with data, with the possible exception of the final block. When RMS finds a data block that is not filled, it attempts to start end-of-file processing. To prevent RMS from misinterpreting a partially-filled block as the final file block, the Files-11 C/D ACP uses I/O operations that combine adjacent ISO 9660 logical blocks into full 512-byte logical blocks.

Interleaving

Interleaving is used to gain efficiency in accessing information by storing sequential information on separate tracks. The OpenVMS file system is not natively compatible with interleaving, but ISO 9660 file extents may be interleaved. That is, ISO 9660 extents may consist of logical block groupings that are separated by interleaving gaps. In order to make the OpenVMS file system compatible with interleaving, the Files–11 C/D ACP treats each of the interleaved logical block groups as an extent.

Undefined Record Format

ISO 9660 CD–ROMs may be mastered without a specified record format because the ISO 9660 media can be mastered from platforms that do not support the semantics of files containing predefined record formats. See the VSI OpenVMS System Manager's Manual for details about mounting media with undefined record formats.

When an ISO 9660-formatted CD–ROM contains information written according to VSI specifications, affected records may include a DIGITAL System Identifier (DSI) in the associated extended attribute records (XAR). This section describes how DIGITAL System Identifiers are recorded on ISO 9660 media and how a DSI is used to encode OpenVMS formatted information on the media. Figure 1.4 illustrates the DSI and FAT structures in an XAR.

If the DSI file identifier field (DSI$FILE_SYSTEM_IDENTIFIER) contains a 0 and the DSI file version field (DSI$FILE_SYSTEM_VERSION) contains either a 1 or a 2, use the area immediately following the DSI to obtain OpenVMS file and record information (See (B), Figure 1.4.)

If the DSI file version field contains a 1, the area immediately following the DSI contains a binary, hex-encoded, file attributes block that provides file and record information. (See (C), Figure 1.4.)

If the DSI file version field contains a 2, the area immediately following the DSI contains an ASCII, hex-encoded, byte stream that provides file and record information. (See (D) in Figure 1.4.)

When the DSI file version field contains a 0, the area immediately following the DSI will not contain file and record information. Nevertheless, if the media is mounted for DSI protection, the OpenVMS UIC codes and permission codes for system, owner, group, and world (SOGW) users will be enforced.

The format of ANSI magnetic tape volumes is based on Level 3 of the ANSI standard for magnetic tape labels and file structure for information interchange. This standard specifies the format, content, and sequence of volume labels, file labels, and file structures. According to this standard, volumes are written and read on 9-track magnetic tape drives only. The contents of labels must conform to prescribed data and record formats. All labels must consist of ASCII “a” characters.

The ANSI magnetic tape format allows you to write binary data in the file sections (see Figure 1.6) of files. However, if you plan to use such files for information interchange across systems, make sure that the recipient system can read the binary data.

The RMS magnetic tape ancillary control process (MTAACP) is the internal operating system software process that interprets the logical format of ANSI magnetic tape volumes. Transparent to your process, the MTAACP process reads, writes, and interprets ANSI labels before passing this information to RMS and $QIO system services. These services, in turn, read, write, and interpret the record format of the data written in the file section.

Figure 1.6 displays the arrangement and function of these components.

Beginning-of-Tape and End-of-Tape Markers

Every volume has beginning-of-tape (BOT) and end-of-tape (EOT) markers. These markers are pieces of photoreflective tape that delimit the writable area on a volume. ANSI magnetic tape standards require that a minimum of 14 feet to a maximum of 18 feet of magnetic tape precede the BOT marker; a minimum of 25 feet to a maximum of 30 feet of magnetic tape, of which 10 feet must be writable, must follow the EOT marker. The EOT marker indicates the start of the end of the writable area of the tape, rather than the physical end of the tape. Therefore, data and labels can be written after the EOT marker.

Tape Marks

Tape marks separate the file labels from the file sections, separate one file from another, and denote the logical end-of-volume. On the basis of the number and relative placement of tape marks written on a volume, OpenVMS systems determine whether a tape mark delimits a label, a file, or a volume.

Tape marks are written both singly and in pairs. Single tape marks separate either a file section from the header and trailer labels or one file from another. When written after a set of header labels, a single tape mark signals the beginning of a file section. When written before a set of trailer labels, a single tape mark indicates the end of a file section. When written after a trailer label set, a single tape mark separates one file from another.

Double tape marks indicate that either an empty file section exists or the logical end-of-volume has been reached. OpenVMS systems create an empty file when a volume is initialized.

Labels

Labels identify, describe, and control access to volumes and their files. The ANSI magnetic tape format supports volume, header, and trailer labels. The volume labels are the first labels written on a volume. They identify the volume and the volume owner and specify access protection. Header and trailer labels are sets of labels that identify, describe, and delimit files. Header labels precede files; trailer labels follow files.

Table 1.6 lists the labels supported by OpenVMS operating systems. All other ANSI magnetic tape labels are ignored on input.

Although each type of label uses a different format to organize its contents, all labels conforming to Version 3 of the ANSI magnetic tape standard must consist of ASCII “a” characters. Some labels contain reserved fields designed for future system use or future ANSI magnetic tape standardization. Reserved fields also must consist of ASCII “a” characters; however, OpenVMS systems ignore these characters on input.

Each of the file and volume configurations is illustrated in the sections that follow.

Single File Residing on a Single Tape Volume

A single file on a single tape volume configuration consists of one file on one volume. The components of the ANSI magnetic tape format for this configuration are illustrated in Figure 1.7.

Single File Requiring Multiple Tape Volumes

A single-file/multivolume configuration consists of one file that spans two or more volumes in a volume set. Figure 1.8 illustrates the components of the ANSI magnetic tape format for this configuration.

Multiple Files on a Single Tape Volume

A multifile/single-volume configuration consists of two or more files on a single volume. It is the most common file and volume configuration. Figure 1.9 illustrates the components of the ANSI magnetic tape format for this configuration.

Multifile/Multivolume Configuration

A multifile/multivolume configuration consists of two or more files that span two or more volumes in the same volume set. Figure 1.10 illustrates the components of the ANSI magnetic tape format for this configuration.

The sections that follow describe the first volume (VOL1) and second volume (VOL2) labels.

OpenVMS operating systems support four file-header labels: HDR1, HDR2, HDR3, and HDR4. The HDR3 and HDR4 labels are optional. The following sections describe and illustrate each file-header label.

filename.type;version

     Input                
 
"AB2&D""FgHI*k4""#-M";2             
 
"##########"                        
 
""""""""""""""""""""""""""";        
 
"DWDEVOP.DAT"                       
 
"VMS_LONG_FILENAME.LONG_FILETYPE"   
 
 
   Output to User Process              
 
"AB2&D""FGHI*K4""#-M";2                 
 
"".;                                    
 
"""""""""""""""""""""""""""".;          
 
DWDEVOP.DAT;                            
 
VMS_LONG_FILENAME.LONG_FILETYPE         
 
 
   Output to HDR1 Label 
 
AB2&D"FGHI*K4"#-M 
 
################# 
 
"""""""""""""#### 
 
DWDEVOP.DAT###### 
 
VMS_LONG_FILENAME

version number = [(generation number - 1) * 100] + generation version-number + 1

generation number = [(version number - 1)/100] + 1

1.3.1.6.2. HDR2 Label

The HDR2 label describes the record format, maximum record size, and maximum block size of a file.

Record Format Field

The record format field specifies the type of record format the file contains. The operating system supports two record formats: fixed length (F) and variable length (D). When files contain record formats that the system does not support, it cannot interpret the formats and classifies them as undefined.

Fixed-length records are all the same length. No indication of the record length is required within the records because either the HDR2 label defines the record length or you specify the record length with the /RECORDSIZE qualifier. A fixed-length record can be a complete block, or several records can be grouped together in a block.

Fixed-length blocked records are padded to a multiple of 4 records. Variable-length records are padded to the block size. If a block is not filled, it will be padded with circumflex characters (^). The standard does not allow records containing only circumflexes; the system will interpret this as padding, not data.

Figure 1.11 shows a block of fixed-length records. Each record has a fixed length of 50 bytes. All six records are contained in a 300-byte block. The records are blocked—that is, grouped together as one entity—to increase processing efficiency; when records are blocked, you can access many of them with one I/O request. The block size should be a multiple of the record size.

The size of a variable-length record is indicated by a record control word (RCW). The RCW consists of four bytes at the beginning of each record. A variable-length record can be a complete block, or several records can be grouped together in a block.

Two variable-length records are shown in Figure 1.12. The first consists of 54 bytes, including the RCW. The second consists of 112 bytes, including the RCW. The records are contained in a 166-byte block.

Do not use system-dependent record formats on volumes used for information interchange. OpenVMS system-dependent formats are stream and variable with fixed-length control (VFC).

Block Length Field

The block length field denotes the maximum size of the blocks. According to the ANSI magnetic tape standard, valid block sizes range from 18 to 2048 bytes. However, the operating system allows you to specify a smaller or larger block size by using the /BLOCKSIZE qualifier with the MOUNT command. To specify the block size using RMS, see the BLS field in the file access block (FAB) in the VSI OpenVMS Record Management Services Reference Manual. When you specify a block size outside the ANSI magnetic tape standard range, the volume may not be processed correctly by other systems that support the ANSI magnetic tape standards.

Record Length Field

The record length field denotes either the size of fixed-length records or the maximum size of variable-length records in a file. Valid RMS record sizes vary, depending on the record format. The range for fixed-length records is 1 to 65,534 bytes; the range for variable-length records is 4 to 9,999 bytes, including the 4-byte RCW. Therefore, the maximum length of the data area of a variable-length record is 9,995 bytes. To comply with ANSI magnetic tape standards, the record size should not be larger than the maximum block size of 2,048 bytes, even though OpenVMS systems allow larger record sizes (when the block size is larger).

For volumes containing files that do not have HDR2 labels, you must specify MOUNT/RECORDSIZE=n at mount time. The variable n denotes the record length in bytes. Files without HDR2 labels were created by a system that supports only level 1 or 2 of the ANSI standard for magnetic tape labels and file structure. Records should be fixed length because this is the only record format that either level supports. If you do not specify a record size, each block will be considered a single record.

Implementation-Dependent Field

The implementation-dependent field contains two 1-character subfields that describe how the operating system interprets record format and form control.

The first subfield, character position 16, denotes whether the RMS attributes are in this label or the HDR3 label. If character position 16 contains a space, the RMS attributes are in the HDR3 label; if it contains any character other than a space, character position 16 is the first byte of the RMS attributes in the HDR2 label. The attributes appear in character positions 16 through 36 and 38 through 50.

The second subfield, the form control field at character position 37, specifies the form control that defines the carriage control applied to records within a file. Possible values supported for RMS magnetic tape volumes are listed below.

A	First byte of record contains Fortran control characters.
M	The record contains all form control information.
space	Line-feed/carriage-return combination is to be inserted between records when the records are written to a carriage-control device, such as a line printer or terminal. If form control is not specified when a file is created, this is the default.

Buffer-Offset Length Field

For implementations that support buffer offsets, the buffer-offset length field indicates the length of information that prefixes each data block. The magnetic tape file system supports the input of buffer offset, which means that the buffer-offset length obtained from the HDR2 label (when reading the file) is used to determine the actual start of the data block. The magnetic tape file system does not support the writing of a buffer offset.

Note that, if you open a file for append or update access and the buffer-offset length is nonzero, the open operation will not succeed.

The operating system supports two sets of trailer labels: end-of-file (EOF) and end-of-volume (EOV). A trailer label is written for each header label.

The particular label that OpenVMS systems write depends on whether a file extends beyond a volume. If a file terminates within the limits of a volume, EOF labels are written to delimit the file (see Figure 1.7). If a file extends across volume boundaries before terminating, EOV labels are written, indicating that the file continues on another volume (see Figure 1.8).

In sequential access mode, storage or retrieval begins at a designated point in the file and continues sequentially through the file. RMS begins accessing records at the start of the file, unless you either specify the starting point explicitly or establish a starting point through a previous operation.

In the sequential access mode, your program issues a series of requests to RMS to retrieve or store succeeding records in a file. Before acting on these requests, RMS checks the file organization to determine how to proceed. The following sections describe how RMS handles sequential access for each of the three file organizations.

Sequential Access to Sequential Files

In a sequential file, records are stored adjacent to one other. To retrieve a particular record within the file, your program must open the file and successively retrieve all records between the current record position and the selected record.

Figure 2.1 shows a disk surface. Each lettered section on the surface represents a record in a sequential file, beginning with record A. When the program requests sequential access to the file records, RMS interprets each request in the context of the file's organization.

Because this particular file is sequential, RMS complies with each request (except for the first request) by accessing the record immediately following the previously accessed record. For example, after RMS accesses record A, it updates the current-record position to record B in anticipation of the next request.

There are limitations imposed by sequential access. When accessing data sequentially, a program can access a previous record only by reopening or rewinding the file, or by switching to a random access mode. (See Chapter 8 for details.) Another limitation of sequential access is that you can add records only to the end of the file.

Sequential Access to Relative Files

Relative files may be accessed sequentially even if some of the fixed-length file cells are empty (because records were never stored in them or because records were deleted from them). RMS ignores empty cells and sequentially searches for the next occupied cell. For example, assume a relative file contains records only in cells 1, 3, and 6. RMS responds to a sequential retrieval request by retrieving the record in cell 1, then the record in cell 3, and then the record in cell 6.

Figure 2.2 shows how RMS checks each cell, ignores an empty cell when it finds one, and then checks the next cell for a record.

When storing records sequentially in a relative file, RMS places each new record in the cell whose relative record number is one higher than the most recently accessed cell, provided the cell is empty. If the cell is not empty, the new record cannot be stored in it. Instead, RMS returns an error status.

As Figure 2.3 shows, the program directs RMS to store record F in cell 2. Record A already occupies cell 1 but cell 2 is empty, so RMS can store the record in this cell. If this request is followed by a request to sequentially store the next record, RMS stores the record in cell 3, which is also empty. However, if the program tries to store a new record in the next cell (which already contains record B), the attempt fails.

Note that although RMS cannot store a new record in a cell that is already occupied, your program can modify the record occupying the cell.

Sequential Access to Indexed Files

When a program sequentially accesses an indexed file, RMS uses one or more indexes to determine the order in which to process the file records. Because index entries are ordered by key values, an index represents a logical ordering of the records in the file. If you define more than one key for the file, each index associated with a key represents a different logical ordering of the records in the file. Your program can then use the sequential access mode to retrieve records in the logical order represented by any index.

To retrieve records sequentially from an indexed file, your program must first specify a key of reference (for example, primary key, first alternate key, second alternate key, and so on). For successive retrievals, RMS uses the appropriate index to retrieve records based on how the records are ordered in the index.

If RMS accesses the index in ascending sort order, it returns the record with a key value equal to or higher than the key value in the previously accessed record. Conversely, if RMS accesses records in descending order, it accesses the next record having a key value equal to or lower than the key value in the previously accessed record. In contrast to a request to retrieve data sequentially from an indexed file, a request to store data sequentially in an indexed file does not require a key of reference. Rather, RMS uses the definition of the primary key to place the record in the primary index and, where applicable, uses the definition of the appropriate alternate key to place a record pointer in the alternate index.

When a program issues a series of requests to sequentially store data, RMS verifies that the key value in each successive record is ordered correctly.

RMS supports random access for all relative files, all indexed files, and a restricted set of sequential disk files—those having fixed-length records. In random access mode, your program (not the file organization) determines the record processing order. For example, to randomly access a record in a relative file or a record in a sequential disk file having fixed-length records, your program must provide the relative record number of the cell containing the record. Similarly, to randomly access a record from an indexed file, your program must provide the appropriate key of reference and key value.

Random Access to Sequential and Relative Files

Unlike sequential access, random access follows no specific pattern. Your program may make successive requests for storing or retrieving records anywhere within the file. In Figure 2.4, the program directs RMS to retrieve the record from the sixth cell in a relative file (record C) and then requests RMS to retrieve record F, which occupies the second cell.

Compare Figure 2.4 withFigure 2.1 andFigure 2.2.

Random Access to Indexed Files

To randomly access a record from an indexed file, your program must specify both a key value and the index that RMS must search (for example, primary index, first alternate key index, and so on). When RMS finds a record with a matching key value, it passes the record to your program.

Your program can use several methods to randomly access a record by key:

Chapter 8 describes these key match conditions in more detail.

In contrast to record retrieval requests, program requests to store records randomly in an indexed file do not require the separate specification of a key value. All keys (primary and any alternate key values) are in the record itself.

When your program opens an indexed file to store a new record, RMS uses the key definitions in the file to find each key field in the record and to determine the length of each key. After writing the new record into the file, RMS uses the record's key values to make appropriate entries in the related indexes so that the record can be accessed subsequently using any of its key values.

Every record on disk has a unique file address—the record file address (RFA)—that provides another way to randomly retrieve records in all types of file organizations.

An important feature of the RFA is that it remains constant as long as the record is in the file. RMS returns the RFA to your program each time the record is retrieved or stored. Your program can either ignore the RFA or keep it as a random-access pointer to the record for subsequent accesses.

Figure 2.5 contains two illustrations. The first shows that when a record is stored, its RFA is returned to the program. The second shows that when the program wants to access the record randomly, it provides RMS with the RFA.

When you specify fixed-length record format, all file records are the same length and each record begins on an even-byte boundary. For example, when you specify 9-byte, fixed-length records, each block can hold up to 51 records (512/10) not 56 records (512/9).

When you accept the block span option (the default), the maximum fixed-length record size for sequential files is 32767 bytes. If you specify no block spanning, the maximum fixed-length record size is 512 bytes (one block). For additional information about selecting the block span option, see the VSI OpenVMS Record Management Services Reference Manual.

The record length set at file-creation time cannot be changed. It becomes part of the information that RMS stores and maintains for the file.

For the fixed-length record format, each record occupies the same amount of space in the file, and the specified length must be able to accommodate the longest record in the file. If any record fields are not used, your program must be able to detect them and provide appropriate error processing. If you specify the block span option, records are limited to 512 bytes.

When you specify the variable-length record format, each record is as long as the data within it requires, except that all records are padded to an even number of bytes. The number of bytes is encoded in a 2-byte count field prefixed to the record.

The field may be coded in either LSB (least significant byte) or MSB (most significant byte) format.

The count field for each record begins on an even-byte boundary and contains the number of bytes in the record. RMS builds the count field from information in your program and treats it separately from the associated record data field.

RMS uses the following types of variable-length record formats:

When you create a file of variable-length records, specify the value (in bytes) of the largest record permitted in the file.

If you take the block span option (the default), the maximum variable-length record size is 32767 bytes. If you specify no block spanning, the maximum variable-length record size is 510 bytes. For additional information about selecting the block span option, see the VSI OpenVMS Record Management Services Reference Manual.

Any attempt to store a record containing more bytes than the specified value results in an error. If you specify a value of 0, any length record can be stored; however, you must consider the bucket capacity limitation defined for relative and indexed files.

Figure 2.6 compares fixed-length record formats and variable-length record formats as they apply to sequential files. Each format shows a portion of a file that contains three records. The comparable record in each format contains the same number of bytes. The first record has 8 bytes, the second, 16, and the third, 24. For the fixed-length record format, the record length is set at 32 bytes. Therefore, RMS considers all 32 bytes to be used.

Clearly, variable-length records can save space; but if records are updated in place, you should consider trading off some space efficiency for update flexibility. All records in a relative file are in fixed-length cells. Here, variable-length records do not save space; in fact, the two count-field bytes prefixing each record actually consume additional space.

In the indexed file organization, the capacity of the data bucket and the maximum record size limit record length.

VFC records are similar to variable-length records except that a fixed-length control field is prefixed to the variable-length data portion. Unlike variable-length records, VFC records cannot be used in indexed files.

When you create a file for VFC records, you must specify the value (in bytes) of the longest record permitted in the file. If you accept the block span option (the default), the maximum VFC record size is 32767 bytes, less the number of bytes in the fixed-length control field. If you specify no block spanning, the maximum VFC record size is 510 bytes, less the number of bytes in the fixed-length control field. For additional information about selecting the block span option, see the VSI OpenVMS Record Management Services Reference Manual.

Any attempt to store a record containing more bytes than the specified value results in an error. If you specify a value of 0, any length record can be stored.

You must also specify the value in bytes of the fixed-length control field. The fixed-length control field lets you include within the record additional data that may have no direct relationship to the other contents of the record. For example, the fixed-length control field may contain line-sequence numbers for every record in the file. The program does not use the line-sequence numbers, but they are helpful in locating records during file editing.

At the VAX MACRO level, you establish the length of the control field for VFC records using the FAB$B_FSZ field in the FAB. The Open, Create, and Display services provide the control field length in the XAB$B_HSZ field of the File Header Characteristic XAB. For more information, see the VSI OpenVMS Record Management Services Reference Manual.

When writing a VFC record to a file, RMS merges the fixed-length control field with the variable-length record data and prefixes the merged record with the count field. Figure 2.7 shows how RMS writes a VFC record to a file.

When RMS reads a VFC record, it uses the count field to determine the overall length of the record, and it uses the appropriate file attribute to determine the length of the control field. After subtracting the control-field length from the overall record length, RMS uses the result to separate the data from the control information. It then processes the data and stores the control information in a designated storage area for program use, if applicable. See Figure 2.8.

RMS supports the stream record format for sequential files on disk devices only. In a stream-formatted file, RMS treats the data as a continuous stream of bytes, without control information. Stream records are always permitted to span block boundaries.

To sequentially retrieve indexed records, your program must specify the key for the first access. RMS then uses the index for that key to retrieve successive records. For example, assume an index file with three records, having primary keys of A, B, and C, respectively. To retrieve these records sequentially in ascending sort order, your program must provide the key A on the first access; RMS accesses the next two records without further key inputs from your program.

To randomly retrieve records in an index file, your program must provide the appropriate key value for each access. Now assume an index file with three records having primary keys A, B, and C that are retrieved in C, A, B order. On the first access, your program must provide the key C, on the next access the key A, and on the final access the keyB.

In an indexed file, each record includes one or more key fields (or simply keys) that RMS uses to build related indexes. Each key is identified by its location, its length, and whether it is a simple or a segmented key.

A simple key may be any one of the following data types:

Segmented keys are fields of character strings having from 2 to 8 segments that may be or may not be contiguous; however, RMS treats all key segments as a logically contiguous string. Segmented keys enhance flexibility in manipulating data files by letting you select the placement of data fields and then tailoring the key structure to fit this layout. You can improve performance by defining a segment that contains the desired key together with another segment that contains a unique field, thereby making the entire key unique. When only noncontiguous portions of a text string are needed for a key, you can improve efficiency by defining smaller keys that include only these segments.

For an indexed file, you must define at least one key, the primary key, and you can optionally define one or more alternate keys. RMS uses alternate keys to build indexes that identify records in alternate sort orders. As with the primary key, each alternate key is defined by location and length.

In addition to defining keys, you can specify various key characteristics (FDL secondary key attributes) including the following:

Key characteristics can be defined separately for each key.

When you do not allow duplicate key values, RMS rejects any attempt to put a record into a file if it contains a key value that duplicates a key value already present in another record. Similarly, when alternate key values cannot be changed, RMS does not allow your program to update a record by changing the alternate key value. If you disallow a null value for a key, RMS inserts an entry for the record in the associated alternate index.

Figure 2.12 illustrates the general structure of an indexed file containing only a primary key: the employee name in an employment record file. Figure 2.13 illustrates the general structure of an indexed file in which the primary key and one alternate key are defined. The primary key is the name of the employee; the alternate key is the employee badge number in an employment record file.

RMS lets you specify either ascending sort order or descending sort order for each key. At the VAX MACRO level, you encode sort order within the key data type field (XAB$B_DTP) of the associated key XAB; you use the attribute KEY TYPE at the FDL level. For example, if you want to build an index of string data type keys in ascending sort order using VAX MACRO, you enter the following line in the associated key XAB:

DTP = STG

To build an index of string data type keys in descending sort order, you enter this line in the associated key XAB:

DTP = DSTG

See the VSI OpenVMS Record Management Services Reference Manual for a complete listing of key data types used to specify ascending and descending sort order.

The RMS multinational key feature lets you assign alternative (non-ASCII) collating sequences to a key. For example, a program can sort records using a key that accesses a collating sequence based on French or alternatively accesses a collating sequence based on Spanish.

The basis for this feature is the National Character Set utility (NCS). When an application program creates an index file with an alternative collating sequence, it calls NCS. NCS responds by retrieving the collating sequence from the NCS library, storing it in local memory and providing the calling program with a pointer to it. In addition to naming the collating sequence, the calling program must provide NCS with a location for storing the pointer (CS_ID) to the memory location of the collating sequence. (For information about NCS, see the OpenVMS National Character Set Utility Manual.)?

When the application program creates the data file, it uses the pointer to copy the collating sequence from local memory into the data file's prolog space. A collating sequence is typically 1 block long.

The application program may specify a collated key from either the RMS interface or the FDL interface.

From the RMS interface, the application program identifies the collating sequence using an appropriate string descriptor and includes a symbolic reference to the location of the pointer. As with all other keys, the application program may specify either ascending or descending sort order. From the RMS interface, you specify the key data type COL for an ascending sort order or the key data type DCOL for descending sort order.

From FDL, you specify a collated key by selecting one of the collated key data types (collated for ascending sort order, decollated for descending sort order) from the INDEXED file script. FDL responds by prompting for the name of the collating sequence. If you enter an invalid collating sequence, any attempt to use the FDL file for creating a data file will be unsuccessful, and NCS generates the following error message:

%NCS-F-NOT_CS, name or id is not a CS

   .
   .
   .
        .TITLE Example 
; 
; Define key type as COL or DCOL 
; 
KEY0:   $XABKEY   
   .
   .
   .
               DTP=COL 
; 
; Descriptor for collating sequence name 
; 
CS_DESC:       .ASCID /Spanish/ 
               .EXTRN NCS$GET_CS 
   .
   .
   .
; Collating sequence name descriptor 
; 
               PUSHAL   CS_DESC 
; 
; Where to store address of collating sequence 
; 
               PUSHAL   KEY0+XAB$L_COLTBL 
; 
; Fetch collating sequence 
; 
               CALLS    #2,G^NCS$GET_CS 
               BLBC     R0,ERROR 
; 
; Create file 
; 
               $CREATE  FAB=OUTFAB 
               BLBC     R0,ERROR

Some advantages and disadvantages of the indexed file organization are outlined in Table 2.5.

When you create a file, you should allocate enough space to store it in one contiguous section of the disk. If the file is contiguous on the disk, it requires only one retrieval pointer in the header; this reduces disk head motion.

You should also consider allocating additional space in anticipation of file growth to reduce the number of required extensions.

You can allocate space either by using the FDL attribute FILE ALLOCATION or by using the file access control block field FAB$L_ALQ.

Use the FILE secondary attribute CONTIGUOUS to arrange the file contiguously on the disk, if you have sufficient space. If you assign the CONTIGUOUS attribute and there is not enough contiguous space on the disk, RMS does not create the file. To avoid this, consider using the FDL attribute BEST_TRY_CONTIGUOUS instead of the CONTIGUOUS attribute. The BEST_TRY_CONTIGUOUS attribute arranges the file contiguously on the disk if there is sufficient space or noncontiguously if the space is not available for a contiguous file.

You can make this choice by accepting the FDL default values for both attributes—NO for CONTIGUOUS, YES for BEST_TRY_CONTIGUOUS or by taking the RMS FAB$V_CBT option in the FAB$L_FOP field.

An extend operation (file extend) adds unused disk blocks to an RMS file when the free space within a file is exhausted. If the unused disk blocks are not contiguous to the previously allocated disk blocks of the file, the file becomes fragmented. As a file becomes fragmented, access time increases and processing performance can degrade. Appropriate use of extend operations can minimize file fragmentation.

If you intend to add large amounts of data to a file over a short time, using large extends will minimize file fragmentation and the overhead of extend operations. Conversely, if you intend to add small amounts of data to a file over a long time, smaller file extends can avoid wasted disk space.

There are two methods for extending files. One method is for an application program to call the $EXTEND service (see the VSI OpenVMS Record Management Services Reference Manual for details). When it calls the $EXTEND service, the application must specify an explicit extend size, in disk blocks, because no defaults are used to determine the extend size.

The other method is for RMS to automatically extend (auto extend) a file when free space is needed. You can specify the size of auto extends using various default extension quantities, or you can have RMS supply a default extend size. However, when RMS supplies a default, it uses an algorithm that allocates a minimal extend. Repeated minimal extends can increase file fragmentation.

Only RMS sequential disk files that have been opened for write access (FAB$V_PUT, FAB$V_UPD, FAB$V_DEL or FAB$V_TRN) can be truncated. This applies to unshared and shared sequential files.

Two types of truncation can occur on RMS sequential files: RMS truncation and Ancillary Control Procedure (ACP) truncation.

RMS truncation involves resetting the end-of-file (EOF) pointer back to a previous position (possibly the beginning) of a sequential file to reuse the allocated space in a file. RMS truncation is described in the VSI OpenVMS Record Management Services Reference Manual under the $TRUNCATE service.

ACP truncation occurs when RMS closes a sequential file and requests that the ACP deallocate all disk blocks allocated beyond the EOF of the file. The primary use of ACP truncation is to conserve disk space. The remainder of this section deals with ACP truncation.

You can also use ACP truncation in conjunction with large extend sizes to reduce disk fragmentation. If a file is growing slowly over time, the application can allocate the largest possible extend, and when finished, it can use ACP truncation to deallocate any unused space at the end of the sequential file. However, if a sequential file is continually growing, excessive ACP truncation can lead to an increase in disk fragmentation resulting in more CPU and I/O overhead.

ACP truncation can be requested directly by way of the programming interface by setting the FAB$V_TEF bit on input to the $OPEN, $CREATE, or $CLOSE service. The ACP truncation occurs on the close of the sequential file. Note that ACP truncation can occur on shared as well as unshared sequential files. If there are shared readers of the file, ACP truncation is postponed until the last reader of the file closes the file. If there are other writers of a shared sequential file, then ACP truncation requests are ignored. However, the ACP truncation request of the last writer to close the file will be honored.

ACP truncation of a sequential file can be automatically requested by RMS if an auto extend has been done during this file access and no file default extend quantity exists to be used for the auto extend. Using ACP truncation in this instance avoids wasting space when auto extending with a less precise extend quantity default, such as the system default extend quantity.

Another file design consideration is to reduce the number of times that RMS must transfer data from disk to memory by making the I/O units as large as possible. Each time RMS fetches data from the disk, it stores the data in an I/O memory buffer whose capacity is equal to the size of one I/O unit. A larger I/O unit makes more records immediately accessible to your program from the I/O buffers.

In general, using larger units of I/O for disk transfers improves performance, as long as the data does not have to be swapped out too frequently. However, as the total space used for I/O buffers in the system approaches a point that results in excessive paging and swapping, increasing I/O unit size degrades system performance.

RMS performs I/O operations using one of the following I/O unit types:

A block is the basic unit of disk I/O, and it consists of 512 contiguous bytes. Even if your program requests less than a block of data, RMS automatically transfers an entire block.

The other I/O units—multiblocks and buckets—are made up of block multiples. A multiblock is an I/O unit that includes up to 127 blocks but whose use is restricted to sequential files. See Section 3.3.2 for details. A bucket is the I/O unit for relative and indexed files and it may include up to 63 blocks. See Section 3.4 and Section 3.5 for details.

For indexed files, another design strategy is to separate the file into multiple areas. Each area has its own extension size, initial allocation size, contiguity options, and bucket size. You can minimize access times by precisely positioning each area on a specific volume, cylinder, or block.

For instance, you can place the data area on one volume of a volume set and place the indexed area on another volume. If your application is I/O bound, this strategy could increase its throughput. You can also derive data bucket contiguity by allocating extra space for future extensions of the data area.

Any attempt to insert a record into a filled bucket results in a bucket split. When a bucket split occurs, RMS tries to keep half of the records (including the new record, if applicable) in the original bucket and moves the remaining records to a newly created bucket.

Excessive bucket splitting can degrade system performance through wasted space, excessive processing overhead, and file fragmentation. For example, each record that moves to a new bucket must maintain a forward pointer in the original bucket. The forward pointer indicates the record's new location. At the new bucket, the record must maintain a backward pointer to its original bucket. RMS uses the backward pointer to update the forward pointer in the original bucket if the record later moves to yet another bucket.

When a program attempts to access a record either by alternate key or by RFA, it must first go to the bucket where the record originally resided, read the pointer to the record's current bucket residence, and then access the record.

To avoid bucket splits, you should permit buckets to be only partially filled when records are initially loaded. This provides your application with space to make additional random inserts without overfilling the affected bucket.

Section 3.5.2.2 describes fill factors in more detail.

When a program accesses a data file, it transfers the file from disk into memory using I/O units of blocks, multiblocks, or buckets. The I/O units are subsequently placed in memory I/O buffers sized to be compatible with the I/O units.

If you do not have enough buffers, excessive I/O transfers may degrade the performance of your application. On the other hand, if you have too many buffers, performance may degrade because of an overly large working set. As a rule, however, increasing the size and number of buffers can improve performance if the data read into the buffers will soon be processed and if your working set can efficiently maintain the buffers. In fact, changing the size and number of buffers is the quickest way to improve the performance of your application when you are processing localized data.

The optimum number of buffers depends on the organization and use of your data files. The recommended way to determine the optimum number of buffers for your application is to use the Edit/FDL utility.

The number of I/O buffers is a run-time parameter you set with the FDL editor by adding the CONNECT secondary attribute MULTIBUFFER_COUNT to the definition file. (See Chapter 9.) With a low-level language, you can set the value directly into the RAB$B_MBF field of the record access block, or you can set the count using the XAB$_MULTIBUFFER_COUNT XABITM if you want to specify more than 127 buffers.

Alternatively, the number of buffers may be specified for a process using the DCL command SET RMS_DEFAULT/BUFFER_COUNT=n, where the variable n represents the desired number of buffers. With this command, you may set distinct values for your sequential, relative, and indexed files using the appropriate file organization qualifier. If you omit the file organization qualifier, sequential organization is assumed. To examine the current settings for the process and system default multibuffer count, use the DCL command SHOW RMS_DEFAULT. If none of the above methods is used, RMS uses the system-wide default value established by the system manager. If the system-wide default is either omitted or is set to 0, RMS uses a value of 1 for sequential and relative files and a value of 2 for indexed files.

For more details about using multiple buffers with sequential files, see Section 3.3.3. For more details about using multiple buffers with relative files, see Section 3.4.2. For more details about using multiple buffers with indexed files, see Section 3.5.2.3.

Chapter 7 describes the use of multiple buffers in the context of shared files.

One way to improve performance through minimized I/O is to use the deferred-write option to keep data in memory as long as practicable. However, you must determine if this added performance benefit is worth the increased risk of losing data if the system crashes before a buffer is transferred to disk.

With indexed files and relative files, you may use the deferred-write option to defer writing modified buckets to disk until the buffer is needed for another purpose or until the file is closed.

Typically, the largest gains in performance come from using the deferred-write option with sequential access. Retrieving and modifying records one after the other permits you to access all of the records from one bucket while the bucket is in memory.

You may also improve performance by using the deferred-write option to prevent writing a shared sequential file to disk on each modification. However, this increases the risk of losing data if the system crashes before the full buffer is transferred to disk.

Note that nonshared sequential files behave as if the deferred-write option is always specified, because a buffer is only written to disk after it is completely filled.

Deferred-write processing is a default run-time option for some high-level languages and can be specified by using clauses in other languages. You can activate this option through FDL by adding the FILE attribute DEFERRED_WRITE. From a low-level language, you can activate the deferred-write option by setting the FAB$V_DFW bit in FAB$L_FOP field.

If several processes are to share a file, you may want to provide the file with global buffers—I/O buffers that two or more processes can access. With global buffers, processes may access file information without allocating dedicated buffers. If you do not allocate dedicated buffers, you can conserve buffer space and buffer management overhead. You gain this benefit at the cost of additional system resources, as described in the VSI OpenVMS Record Management Services Reference Manual.

When you create a file, you can assign the desired number of global buffers by using the FDL editor to set the value for the FILE secondary attribute GLOBAL_BUFFER_COUNT. From a low-level language, you can optionally set the value directly into the FAB$W_GBC field. Alternatively, you may use the DCL command SET FILE/GLOBAL_BUFFERS to set the global buffer count.

Global buffers are not used directly to retain modified information when the deferred-write option is enabled. If a global buffer is modified and the deferred-write option is enabled, the contents of the global buffer are copied to a process local buffer before other processes are allowed to access the global buffer contents. Subsequent use of the modified buffer by the process that deferred the writeback refer to the process local buffer while it remains in the process local cache. Reference to the global buffer by another process causes the contents of the process local buffer to be written back to disk.

If a global buffer is modified and the deferred-write option is not enabled, then the contents are written out to disk from the global buffer. Therefore, using global buffers along with the deferred-write option may cause a slight increase in processing overhead if in fact no further references to the modified buffer occur before it is flushed from the cache anyway. For that reason, you may want to disable the deferred-write option for processes that do not reaccess buffers after records have been written to them.

Section 3.3, Section 3.4, and Section 3.5 discuss the use of global buffers in tuning the various file types.

The operation of sequentially organized files can be improved by implementing read-ahead and write-behind processing. These features improve performance by permitting record processing and I/O operation to occur simultaneously. The read-ahead and write-behind features are default run-time attributes in some languages, but they must be explicitly specified in others.

You implement read-ahead and write-behind processing by using two buffers. The processing program uses one buffer, and the I/O subsystem uses the other. In read-ahead processing, the program reads data from one buffer as the second buffer inputs data from the disk. In write-behind processing, one buffer accepts output data from the program, while the second buffer outputs program data to a disk.

The next section provides additional information about read-ahead and write-behind processing.

RMS places information concerning file attributes, key descriptors, and area descriptors in the prolog. You can examine the prolog with the Analyze/RMS_File utility described in Chapter 10.

There are three types of prologs: Prolog 1, Prolog 2, and Prolog 3.

Prolog 1 and Prolog 2 Files

Any indexed file created with a version of the operating system lower than Version 3.0 is either a Prolog 1 file or a Prolog 2 file. Prolog 1 files and Prolog 2 files operate identically.

If an indexed file uses only string data-type keys, the file is a Prolog 1 file.

If an indexed file uses numeric type keys, it is a Prolog 2 file.

You cannot use the Convert/Reclaim utility on a Prolog 1 file or a Prolog 2 file to reclaim empty buckets. If your file undergoes a large number of deletions (resulting in empty, unusable buckets), you must use the Convert utility (CONVERT) to reorganize the file. (Note that CONVERT establishes new RFAs for the records.)

The compression allowed with Prolog 3 files is not possible with Prolog 1 or Prolog 2 files.

Prolog 3 Files

Prolog 3 files can accept multiple (or alternate) keys and all data types (including the nonstring 8-byte BIN8 and INT8 types). They also give you the option of saving space by compressing your data, indexes, and keys.

Key compression compresses the key values in the data buckets. Likewise, index compression compresses the key values in index buckets, and data compression compresses the data portion of the records in the data buckets.

Key or index compression is restricted to the string key data type and the string must be at least 6 bytes in length.

With key or index compression, repeating leading and trailing characters are compressed. With front key compression, any characters that are identical to the characters at the front of the previous key are compressed. For example, the keys JOHN, JOHNS, JOHNSON, and JONES appear as JOHN, S, ON, and NES.

With rear key compression, any repeating characters at the end of the key are compressed to a single character. For instance, the key JOHNSON00000 appears as JOHNSON0.

Enabling index compression results in RMS doing a sequential search in index buckets rather than its default binary search, since each index key value must be expanded until a match is found.

With data compression, RMS can compress sequences of up to 255 repeating characters in the data portion of the user data records. For optimal performance, RMS does not compress sequences having less than five repeating characters.

Compression has a direct effect on CPU time and disk space. Compression increases CPU time, but the keys are smaller, so your application can scan more quickly through the data and index buckets.

The disk space saved by using Prolog 3 indexed files can significantly improve performance. With compression, each I/O buffer can hold more information to improve buffer space efficiency. Compression can also decrease the number of index levels, which decreases the number of I/O operations per random access.

Prolog 3 files can have segmented primary keys, but the segments cannot overlap. If you want to use a Prolog 3 file in this case, consider defining the overlapping segmented key as an alternate key and choosing a different key to be the primary key. If you want to use overlapping primary key segments, you must use a Prolog 2 file.

If record deletions result in empty buckets in Prolog 3 files, you can use the Convert/Reclaim utility to make the buckets usable again. Because CONVERT/RECLAIM does not create a new file, RFAs remain the same.

Note that RMS–11 does not support Prolog 3 files. To use a Prolog 3 file with RMS–11 you must first use the Convert utility to transform the file into a Prolog 1 file or into a Prolog 2 file.

The primary index structure consists of the file's data records and a key pathway based on the primary key (key 0). The base of a primary index structure is the data records themselves, arranged sequentially according to the primary key value. The data records are called level 0 of the index structure.

The data records are grouped into buckets, which is the I/O unit for indexed files. Because the records are arranged according to their primary key values, no other record in the bucket has a higher key value than the last record in that bucket. This high key value, along with a pointer to the data bucket, is copied to an index record on the next level of the index structure, known as level 1.

The index records are also placed in buckets. The last index record in a bucket itself has the high key value for its bucket; this high key value is then copied to an index record on the next higher level. This process continues until all of the index records on a level fit into one bucket. This level is then known as the root level for that index structure.

Figure 3.1 is a diagram of an index structure.

Figure 3.2 illustrates a primary index structure. (For simplicity, the records are assumed to be uncompressed, and the contents of the data records are not shown.) The records are 132 bytes long (including overhead), with a primary key field of 6 bytes. Bucket size is one block, which means that each bucket on Level 0 can contain three records. You calculate the number of records per bucket as shown by the following algorithm:

Substituting the values in this instance:

Note that you must round the result down to the next lower integer, in this case, the integer 3.

Because the key size is small and the database in this example consists of only 27 records, the index records can all fit in one bucket on level 1. The index records in this example are 6 bytes long. Each index record has one byte of control information. In this example, the size of the pointers is 2 bytes per index record, for a total index record size of 9 bytes. You calculate the number of records per bucket in this case as follows:

Again, you must round the remainder down to the next lower integer, 55.

To read the record with the primary key 14, RMS begins by scanning the root level bucket, looking for the first index record with a key value greater than or equal to 14. This record is the index record with key 15. The index record contains a pointer to the level 0 data bucket that contains the records with the keys 13, 14, and 15. Scanning that bucket, RMS finds the record (see Figure 3.3).

Alternate indexes (also referred to as secondary indexes) provide your program with alternate record processing. If you have one or more alternate indexes, you can process data records using any of the alternate keys in addition to processing data with the primary key. Note that a file with alternate indexes does require additional disk space.

The alternate index structure is similar to the primary index structure except that, instead of containing data records, alternate indexes contain secondary index data records (SIDRs). An SIDR includes an alternate key value from a data record in the primary index and one or more pointers to data records in the primary index. (SIDRs have pointers to more than one record only if you allow duplicate keys and there are duplicate key values in the database.) You do not need an SIDR for every data record in the database. If a variable-length record is not long enough to contain a given alternate key, an SIDR is not created. For example, if you define an alternate key field to be bytes 10 through 20 and you insert a 15-byte record, no SIDR is created in that alternate index structure.

When you create a file, you can use null key values to improve performance for programs that use alternate keys. When a secondary index has relatively few entries, performance may diminish because RMS tries to treat the null entries (typically blank keys) as duplicates. The resultant duplicate-key processing is unnecessary and can be avoided by assigning a null key value for the index. By using a null key value, you minimize the list of duplicates. This can improve performance when you insert records because the null key entries do not get processed as index entries. Note that when you sequentially access records using null key processing, null records are not processed.

If you use the string data type, RMS uses the ASCII null character as the default null key value. However, you can specify any character as the null value. If you use numeric keys, RMS uses zero (0) as the null value.

Records in an indexed file can be fixed-length records or variable-length records. Fixed-length records begin with a record header. Variable-length records include a record header followed by a 2-byte count field that contains the number of data bytes in the record. Unlike variable-length records in relative files, each variable-length record in an indexed file requires only enough space for the record. See Table 2.2 for more information on record overhead.

Records cannot span bucket boundaries.

For Prolog 3 files, the maximum record size is 32,224 bytes. For Prolog 1 files and Prolog 2 files, the maximum length for a fixed-length record is 32,234 bytes; the maximum length for a variable-length record is 32,232 bytes. Note that when you specify a record length for a Prolog 3 file that is greater than the maximum record length, RMS automatically converts the file to a Prolog 1 or Prolog 2 file.

Record length should reflect application requirements. There is no advantage to using a record length that is based on the number of bytes in a bucket.

The value of the primary key must be contained within the records. The records can contain either a valid key field value for the alternate keys or, if you specify that null keys are allowed, a field of null characters.

A key is a record field that identifies the record to help you retrieve the record. There are two types of keys—primary keys and alternate keys. Data records are filed in the order of their primary key. The most time-efficient value for primary keys is a unique value that begins at byte 0 of the record. You can allow duplicate keys in the primary index, but duplicate keys may slow performance.

The primary key and alternate keys can be character strings or numerical values. Key type is specified by the FDL attribute KEY TYPE.

If it is not possible to put the records into the file in order of their primary key, you should specify that the buckets not be filled completely when the file is loaded. If you attempt to write a record to a full bucket, a bucket split occurs. RMS keeps half of the records in the original bucket and moves the other records to the newly created bucket. Each time a record moves to a new bucket, it leaves behind a forwarding pointer called a record reference vector (RRV). You should avoid bucket splits because they use additional disk space and CPU time. An extra I/O operation is required to access a record in a split bucket when the program accesses a record by an alternate key or by RFA.

Alternate keys have a direct impact on I/O operations, CPU time, and disk space. The number of I/O operations and the CPU time required for Put, Update, and Delete operations are directly proportional to the number of keys. For example, inserting a record with a primary key and three alternate keys takes approximately four times longer than inserting a record with only a primary key.

To update the value of an alternate key, you have to traverse the alternate index structure twice, and bucket splits are more likely to occur. Randomly accessing an alternate key generally requires an extra I/O operation over a comparable access by the primary key, and extra disk space is required to store each alternate index structure.

Alternate keys are more likely than primary keys to have duplicate values. For example, the zip code is a common alternate key. However, allowing many duplicate values can have a performance cost. Duplicate values can cause clustered record or pointer insertions in data buckets, long sequential searches, a large number of I/O operations, and loss of physical contiguity due to continuation buckets (especially for the primary key).

Where possible, you should validate record keys before inserting the record, especially when you have primary and alternate keys.

In general, as the number of keys increases, so does the time it takes to add and delete records from your file. If CPU time is a critical resource on your system, you should define as few keys as possible.

If you are reading records in your file, the number of keys has relatively little impact on performance.

An area is a portion of an indexed file that RMS treats as a separate entity. You can divide an indexed file into separate areas where each area has its own bucket size, initial allocation, extension size, and volume positioning, just as if each area were a separate file.

Using multiple areas has distinct advantages. However, if each area has a different bucket size, all buffers are as large as the largest bucket. If you use multiple areas, the file itself is probably not contiguous; however, you can make each area within the file contiguous by specifying the FDL attribute AREA CONTIGUOUS. To ensure that the area is created without error, use the AREA BEST_TRY_CONTIGUOUS attribute.

When you separate key and data areas, you tend to keep related buckets close together, thereby decreasing disk seek time. You also minimize the number of disk-head movements for a series of operations. For example, if you have a dedicated multidisk volume set, you could place the data level of a file in an area on one disk and the index levels of the file in an area on a separate disk. Then there is little or no competition for the disk head on the disk that contains the index structures.

One strategy is to allocate a separate area for level 0 of a primary index (the data level). These buckets are the only ones referenced when you access the records sequentially by their primary key, so keeping them in a separate area optimizes that type of operation.

Do not allocate separate areas for level 1 of an index and the other index levels if the index has just one level. In such a case, you force RMS to create an additional level in the index structure.

In most cases, you should allocate at least one area for each alternate index structure. By default, EDIT/FDL creates two areas in an indexed file for each index structure—one for the data level and one for all of the index levels. You can allocate up to 255 areas, so with most applications you can set up enough areas to handle all alternate index structures.

It is possible to set up a separate area for each of the following:

Be sure to allocate sufficient space for each area and to specify area contiguity, because extending an area generally creates a noncontiguous area extent. The resulting noncontiguous extent may be anywhere on the disk, and you may lose the benefits of multiple areas.

If you are using a single area for the file, you should allocate enough contiguous space at creation time for the entire file. If you plan to add data to the file later, you should allocate extra space using the FDL attribute FILE ALLOCATION. To ensure contiguous allocation, set the FDL attribute FILE CONTIGUOUS to YES.

If you are using multiple areas, you should allocate each one by specifying a value for the FDL attribute AREA ALLOCATION.

If the file is relatively small, or if you know that it needs to be extended, you do not have to use multiple areas. In such cases, it is more important to calculate the proper extension size.

To specify multiple areas using an FDL file, you assign each area its own AREA primary attribute. The AREA primary attribute takes as an argument a number whose value identifies the area.

Use the KEY attributes DATA_AREA, LEVEL1_INDEX_AREA, and INDEX_AREA to match each area specified with its index level. In the primary index structure, the primary attribute KEY must take the value 0. Within the KEY 0 section, you assign the DATA_AREA attribute the number which identifies the data record area.

Then you associate the KEY LEVEL1_INDEX_AREA attribute with an AREA by assigning the appropriate area number to the LEVEL1_INDEX_AREA attribute. You also assign the appropriate area number to the INDEX_AREA attribute for the other index levels in the primary index structure. For each alternate index structure, you use the same attributes (DATA_AREA, LEVEL1_INDEX_AREA, INDEX_AREA) in another KEY primary attribute. In KEY sections that define alternate keys, the DATA_AREA is where RMS puts the SIDRs.

For indexed files, the bucket size controls the number of levels in the index structure, which has the greatest impact on performance for most applications. You can specify the bucket size with the FDL attribute FILE BUCKET_SIZE or the control block fields FAB$B_BKS and XAB$B_BKZ. When you sequentially access files, large buckets are generally beneficial.

For keyed access to index files, set the bucket size so that the number of index levels does not exceed four. In general, the smaller the bucket size, the deeper the tree structure. If you find that a small increase in bucket size eliminates one level, use a larger bucket size. At some point, however, the benefit of having fewer levels is offset by the cost of scanning through the larger buckets.

As a rule, you should never increase bucket size unless the increase reduces the number of levels. For example, you may find that a bucket size of 4 or more yields an index with four levels, and a bucket size of 10 or more yields an index with three levels. In this case, you never want to specify a bucket size of 9 because that does not reduce the number of levels, and performance does not improve. In fact, such a specification could hurt performance because each I/O operation takes longer, yet the number of accesses remains the same. However, larger bucket sizes always improve performance if you are accessing the records sequentially by primary key because more records fit in a bucket.

Conversely, with smaller buckets you have to search fewer keys. So if you can cache your whole structure (except for level 0), you can save a lot of time. Also, performance in this case is comparable to flat file design although add operations may take a little longer.

You can decrease the depth of your index structure in two ways. First, you can increase the number of records per bucket by increasing the bucket size, increasing the fill factor, using compression, or decreasing the size of keys and records.

However, changing the bucket size also has disadvantages. Larger buckets use more buffer space in memory. And the number of records per bucket determines bucket search time, which directly affects CPU time. A larger fill factor decreases the room for growth in the file, so bucket splits may occur. Compression increases the record search time.

Alternatively, you can reduce the index depth by decreasing the number of records in the file.

If you are using multiple areas, you can set a different bucket size for each area. You should use different bucket sizes if you are performing random accesses of records in no predictable pattern and if the data records are large. Using different bucket sizes allows you to specify a smaller size for the index structures and SIDRs than for the primary data level.

You can use the Edit/FDL utility to determine the optimum bucket size.

Use the same bucket size for all areas if the data records are small or if the record accesses follow a clustered pattern, that is, if the records that you access have keys that are close in value.

In general, increasing the bucket size increases other resources:

Conversely, decreasing the bucket size decreases the pages per bucket and the average number of keys searched while traversing the tree.

If you know that the application makes random insertions into the database, you should reserve some space in the buckets when records are first loaded into the file. You can specify a fill factor from 50% to 100%. For example, a fill factor of 50% means that RMS writes records in only half of each bucket when the records are first loaded, leaving the remainder of the bucket empty for future write operations. This fill factor minimizes the number of bucket splits.

The fill factor is set with the FDL attributes KEY DATA_FILL and KEY INDEX_FILL. The value assigned to both attributes should be the same.

When you specify a fill factor, consider the following:

At run time, you can specify the number of buffers with the FDL attribute CONNECT MULTIBUFFER_COUNT, the control block field RAB$B_MBF, or the XAB$_MULTIBUFFER_COUNT XABITM. The number of buffers each application needs depends on the type of record access your application performs.

The minimum number of buffers for indexed files is two. If the application performs sequential access on your database, two buffers are sufficient. More than two buffers for sequential access could actually degrade performance. During a sequential access, a given bucket is accessed as many times in a row as there are records in the bucket. After RMS has read the records in that bucket, the bucket is not referenced again. Therefore, it is unnecessary to cache extra buckets when accessing records sequentially.

When you access indexed files randomly, RMS reads the index portion of the file to locate the record you want to process. RMS tries to keep the higher-level buckets of the index in memory; the buffers for the actual data buckets and the lower level index buckets tend to be reused first when other buckets need to be cached. Therefore, you should use as many buffers as your process working set can support so you can cache as many buckets as possible.

When you access records sequentially, even after you have located the first record randomly, you should use a large bucket size. A small multibuffer count, such as the default of two buffers, is sufficient.

If you process your data file with a combination of the above access modes, you should compromise on the recommended bucket sizes and number of buffers.

When you add records to an indexed file, consider choosing the deferred-write option (FDL attribute FILE DEFERRED_WRITE; FAB$L_FOP field FAB$V_DFW). With this option, the buffer into which the records have been moved is not written to disk until the buffer is needed for other purposes, the Flush service is used, or until the file is closed. The deferred-write option, however, may cause records to be lost if a system crashes before RMS transfers the records to the disk.

In general, you must consider several trade-offs when you set the number of buffers your application needs:

With indexed files, buckets (not blocks) are the units of transfer between the disk and memory. You specify the bucket size when you create the file, although you can change the bucket size of an existing file with the Convert utility (see Chapter 10).

If several processes share the indexed file concurrently, you may want to specify that the file use global buffers. A global buffer is an I/O buffer that two or more processes can access. If two or more processes request the same information from a file, each process can use the global buffers instead of allocating its own.

Only one copy of the buffers resides at any one time in memory although the buffers are charged against each process's working set size.

The guideline for using global buffers is the same as the guideline for using local process I/O buffers. Global buffers only provide significant benefits if more than one process refers to the same bucket in the global cache. If bucket contention is high, I/O transfers can be minimized and performance improved. However, global buffers do not always improve performance. For example, multiple processes independently reading records and using sequential access are most apt to refer to separate buckets. In that case, bucket contention is low and the number of I/O transfers is not reduced, so global buffers do not improve performance.

The deferred-write option is a run-time option that can improve performance. It is the default operation for some high-level languages and can be specified by clauses in other high-level languages.

If there is no language support, you can call a VAX MACRO subroutine that sets the FAB$L_FOP field, the FAB$V_DFW option.

When you select the deferred-write option, RMS delays writing a modified bucket to the disk until the buffer is needed to read another bucket into the cache or until another process needs to reference the modified bucket. If a subsequent operation references the bucket before it is flushed out to disk, then one I/O operation has been eliminated. Typically, the largest performance gains come from using the deferred-write option with sequential access because random accesses of the file usually result in several I/O operations to bring in the single records.

Not all operations on indexed files can be deferred. Any operation that causes a bucket split forces the writeback of the modified buckets to disk. (This forced writeback decreases the chances of lost information should a system failure occur.)

Using the deferred-write option improves performance if you are performing multiple I/O operations on a bucket. Consider the following example. The indexed file has a single key and its records are 100 bytes long. The bucket size is 3 blocks with a fill factor of 67%. Thus, there is an average of 10 records in each bucket. A batch program reads each record and updates part of it, beginning at the first record in the file and moving through the records sequentially. Without the deferred-write option, 11 disk I/O operations occur for every 10 records—one to read the bucket and one to write the bucket for each record. With the deferred-write option, only two disk I/O operations occur for every 10 records—one to read the bucket and one to write the bucket after the record operations are completed.

The distributed lock manager allows several users to share files concurrently in an organized manner. RMS uses the lock manager to control file access.

The lock-mastering node controls the record and bucket locking for a given file for users on every node of the OpenVMS Cluster. Initially, it is the first node from which the file is opened. However, another node may become the lock-mastering node when a node either joins or leaves the OpenVMS Cluster.

The lock-mastering node may also change every time the file is opened. When another process opens the file (provided that the file was closed), the node on which that process resides becomes the new lock-mastering node for that file.

Lock requests issued by processes on the lock-mastering node incur less cost than lock requests issued from other nodes. Conversely, the lock-mastering node has the additional work of processing lock requests for that file for all other nodes.

The lock-requesting node is any node in the OpenVMS Cluster other than the lock-mastering node for a given file.

RMS locks buckets and records during record operations only if the file is open for shared writing. Conversely, RMS does no locking during record operations if the file is open for shared read-only access or for exclusive access.

Lock requests for root locks (top-level or parent locks) in an OpenVMS Cluster may be slightly slower than on a single-node system. However, these locks are used when you open and close files, so the time for lock operations is only a fraction of the total time needed to open and close files.

There is no performance difference between a single-node system and an OpenVMS Cluster if the file sharing takes place on a single node of the OpenVMS Cluster. Only when sharing spans across the OpenVMS Cluster nodes does distributed locking occur.

As a result, the record locking itself may take a little longer, but because you have multiple CPUs in the OpenVMS Cluster, your application benefits from the added processing power.

Sharing files in an OpenVMS Cluster also requires enough memory for nonpaged pool to store additional lock data structures. This requirement, however, is dependent upon your processing load.

Sharing files in an OpenVMS Cluster environment also means sharing resources, such as disks and other pieces of I/O hardware. When applications on many nodes share data on one disk, OpenVMS Cluster performance may degrade due to excessive I/O operations.

The FAB is made up of fields that describe various file characteristics and contain the following file-related information:

The FAB lets you use both the creation-time characteristics and the run-time characteristics of RMS. You must define one FAB for each file your program opens or creates.

For more information about the FAB, see the VSI OpenVMS Record Management Services Reference Manual.

Extended attribute blocks (XABs) are optional control blocks that contain supplementary file-attribute information. The following is a partial list of XABs that can be used to provide supporting file information:

Like FABs, XABs allow you to use both the creation-time characteristics and the run-time characteristics of RMS.

With XABs, you can define various file attributes beyond those specified in the associated FABs.

For more information about the extended attribute blocks, see the VSI OpenVMS Record Management Services Reference Manual.

You can use the Edit/FDL utility in two ways: with a terminal dialog (interactively) or without one (noninteractively).

If you use the Edit/FDL utility noninteractively, you can execute only the OPTIMIZE script. The OPTIMIZE script lets you optimize an existing FDL file without an interactive session. For more information, see Section 10.3.

Alternatively, if you use the Edit/FDL utility interactively, you can use all the scripts, each of which has a series of menus. When you invoke the Edit/FDL utility, it displays a main menu. To select a menu item, you only need to enter the first letter of the item because each selection has a unique first letter.

Table 4.1 summarizes the Edit/FDL utility commands.

Ctrl/Z is equivalent to the EXIT command if you use it at the main menu level. If you use it from any other level, Ctrl/Z returns you to the main menu level.

In most cases, a command from the main menu brings up a second level menu. For instance, typing the ADD command displays the following menu:

                    Legal Primary Attributes

ACCESS  attributes set the run-time access mode of the file
AREA x  attributes define the characteristics of file area x
CONNECT attributes set various RMS run-time options
DATE    attributes set the date parameters of the file
FILE    attributes affect the entire RMS data file
KEY y   attributes define the characteristics of key y
NETWORK attributes set run-time network access parameters
RECORD  attributes set the non-key aspects of each record
SHARING attributes set the run-time sharing mode of the file
SYSTEM  attributes document operating system-specific items
TITLE     is the header line for the FDL file

Enter desired primary     (Keyword)[FILE] :

One of the most important features of the Edit/FDL utility is that it helps you create FDL files that define indexed, relative, and sequential data files. To do this, the Edit/FDL utility provides seven scripts that guide you through an interactive session. You can choose one of these scripts at the start of a session, or you can instruct the Edit/FDL utility to automatically invoke a particular script each time that you enter the EDIT/FDL command.

Table 4.2 lists the seven scripts.

An interactive session is controlled by these Edit/FDL utility scripts. You can invoke a script in two ways:

Example 4.1 shows a sample session with the Edit/FDL utility.

                  OpenVMS FDL Editor
 
  Add     to insert one or more lines into the FDL definition
  Delete  to delete one or more lines from the FDL definition
  Exit    to leave the FDL Editor after creating the FDL file
  Help    to obtain information about the FDL Editor
  Invoke  to initiate a script of related questions
  Modify  to change existing line(s) in the FDL definition
  Quit    to abort the FDL Editor with no FDL file creation
  Set     to specify FDL Editor characteristics
  View    to display the current FDL Definition
  Main Editor Function            (Keyword)[Help] : INVOKE
 
 
                Script Title Selection
 
  Add_Key       modeling and addition of a new index's parameters 
  Delete_Key    removal of the highest index's parameters
  Indexed       modeling of parameters for an entire Indexed file
  Optimize      tuning of all indexes' parameters using file statistics
  Relative      selection of parameters for a Relative file
  Sequential    selection of parameters for a Sequential file
  Touchup       remodeling of parameters for a particular index
  Editing Script Title            (Keyword)[-] : INDEXED
  Target disk volume Cluster Size (1-1Giga)[3]    : 3
  Number of Keys to Define        (1-255)[1]      : 1
 
  Line    Bucket Size vs Index Depth      as a 2 dimensional plot
  Fill    Bucket Size vs     Load Fill Percent     vs Index Depth
  Key     Bucket Size vs         Key Length        vs Index Depth
  Record  Bucket Size vs        Record Size        vs Index Depth
  Init    Bucket Size vs Initial Load Record Count vs Index Depth
  Add     Bucket Size vs  Additional Record Count  vs Index Depth
  Graph type to display           (Keyword)[Line] : LINE
  Number of Records that will be Initially Loaded
  into the File                   (0-1Giga)[-]  : 100000
  (Fast_Convert NoFast_Convert RMS_Puts)
  Initial File Load Method        (Keyword)[Fast] : FAST
  Number of Additional Records to be Added After
  the Initial File Load           (0-1Giga)[0]    : 0
 
 
 
 
 
 
  Key  0 Load Fill Percent        (50-100)[100]   : 100
  (Fixed Variable)
  Record Format                   (Keyword)[Var]  : VARIABLE
  Mean Record Size                (1-32229)[-]    : 80
  Maximum Record Size             (0,80-32229)[0] : 0
 
 
 
 
 
 
  (Bin2 Bin4  Bin8  Int2  Int4  Int8  Decimal  String  Collated
      Dbin2 Dbin4 Dbin8 Dint2 Dint4 Dint8 Ddecimal Dstring Dcollated)
  Key  0 Data Type                (Keyword)[Str]  : STRING
  Key  0 Segmentation desired     (Yes/No)[No]    : NO
  Key  0 Length                   (1-255)[-]      : 9
  Key  0 Position                 (0-32220)[0]    : 0
  Key  0 Duplicates allowed       (Yes/No)[No]    : NO
  File Prolog Version             (0-3)[3]        : 3
  Data Key Compression desired    (Yes/No)[Yes]   : YES
  Data Record Compression desired (Yes/No)[Yes]   : YES
  Index Compression desired       (Yes/No)[Yes]   : YES
 
        *|
        9|
        8|
  Index 7|
        6|
  Depth 5|
        4|
        3|  3 3
        2|      2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
        1|                                                  1 1 1 1 1 1 1 1
         +- + - - - + - - - - + - - - - + - - - - + - - - - + - - - - + - +
(25)            1       5        10        15        20        25        30  32
                          Bucket Size (number of blocks)
 
   PV-Prolog Version       3 KT-Key  0 Type     String EM-Emphasis  Flatter ( 3)
   DK-Dup Key  0 Values   No KL-Key  0 Length        9 KP-Key  0 Position      0
   RC-Data Record Comp    0% KC-Data Key Comp       0% IC-Index Record Comp   0%
   BF-Bucket Fill       100% RF-Record Format Variable RS-Mean Record Size    80
   LM-Load Method  Fast_Conv IL-Initial Load    100000 AR-Added Records        0
          (Type "FD" to Finish Design)       
(26)          Which File Parameter (Mnemonic)[refresh]     : FD
(27)  Text for FDL Title Section      (1-126 chars)[null]   
  : FDL_SESSION_EXAMPLE
(28)  Data File file-spec             (1-126 chars)[null]   
  : EXAMPLE.DAT
(29)  (Carriage_Return Fortran None Print)       
  Carriage Control                (Keyword)[Carr] : CARRIAGE_RETURN
 
  Emphasis Used In Defining Default:      (    Flatter_files   )
  Suggested Bucket Sizes:                 (      3     3    27 )
(30)  Number of Levels in Index:              (      2     2     1 )
  Number of Buckets in Index:             (     72    72     1 )
  Pages Required to Cache Index:          (    216   216    27 )
  Processing Used to Search Index:        (    168   168   766 )
(31)  Key  0 Bucket Size              (1-63)[3]       : 3
(32)  Key  0 Name                     (1-32 chars)[null]    
  : SSNUM
(33)  Global Buffers desired          (Yes/No)[No]    : NO
(34)  The Depth of Key  0 is Estimated to be No Greater  
  than 2 Index levels, which is 3 Total levels.
(35)  Press RETURN to continue (^Z for Main Menu) 
 
                    OpenVMS FDL Editor
 
  Add     to insert one or more lines into the FDL definition
  Delete  to delete one or more lines from the FDL definition
  Exit    to leave the FDL Editor after creating the FDL file
(36)  Help    to obtain information about the FDL Editor
  Invoke  to initiate a script of related questions
  Modify  to change existing line(s) in the FDL definition
  Quit    to abort the FDL Editor with no FDL file creation
  Set     to specify FDL Editor characteristics
  View    to display the current FDL Definition
(37)  Main Editor Function            (Keyword)[Help] : EXIT
(38)  DISK$:[FOX.RMS]FDL_SESSION_EXAMPLE.FDL;1  40 lines

Key compression and index compression are not acceptable options when you select a collated key data type.

When the Edit/FDL utility creates an FDL file, it groups the attributes into major sections. The section headings are called primary attributes, and the attributes within a primary section are called secondary attributes. Certain secondary attributes contain a third level of attributes called qualifiers.

The objective of using the Edit/FDL utility is to create an FDL file with optimum values for the various attributes. An FDL file contains a list of the primary and secondary attributes with related qualifiers. If a primary or secondary attribute does not appear in the FDL file, it is assigned its default value.

IDENT
" 1-MAR-1993 14:07:46   OpenVMS FDL Editor"

SYSTEM
        SOURCE                  VMS
FILE
        GLOBAL_BUFFER_COUNT     0
        NAME                    DISK$RMS:[RMSTEST]INDEXED.DAT;3
        ORGANIZATION            indexed
        OWNER                   [RMS1,TEST]
        PROTECTION              (system:RWED, owner:RWED, group:RE, world:)

RECORD
        BLOCK_SPAN              yes
        CARRIAGE_CONTROL        none
        FORMAT                  variable
        SIZE                    2048

AREA 0
        ALLOCATION              233
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             5
        EXTENSION               60
AREA 1
        ALLOCATION              5
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             5
        EXTENSION               5

AREA 2
        ALLOCATION              18
        BEST_TRY_CONTIGUOUS     yes
        BUCKET_SIZE             3
        EXTENSION               6

KEY 0
        CHANGES                 no
        DATA_AREA               0
        DATA_FILL               100
        DATA_KEY_COMPRESSION    no
        DATA_RECORD_COMPRESSION no
        DUPLICATES              no
        INDEX_AREA              1
        INDEX_COMPRESSION       no
        INDEX_FILL              100
        LEVEL1_INDEX_AREA       1
        NAME                    "NUM"
        NULL_KEY                no
        PROLOG                  3
        SEG0_LENGTH             8
        SEG0_POSITION           0
        TYPE                    bin8

KEY 1
        CHANGES                 yes
        DATA_AREA               2
        DATA_FILL               100
        DATA_KEY_COMPRESSION    yes
        DUPLICATES              yes
        INDEX_AREA              2
        INDEX_COMPRESSION       yes
        INDEX_FILL              100
        LEVEL1_INDEX_AREA       2
        NAME                    "NAME"
        NULL_KEY                yes
        NULL_VALUE              0
        SEG0_LENGTH             39
        SEG0_POSITION           9
        TYPE                    string

When you want to create an FDL file, you invoke the Edit/FDL utility with a DCL command in the following form:

EDIT/FDL/CREATE fdl-filespec

The /CREATE qualifier specifies that you want to create an FDL file with the name entered in the fdl-filespec parameter. When the Edit/FDL utility displays the main menu, select the INVOKE command. In response to the INVOKE command, the Edit/FDL utility prompts you for a script. The only appropriate scripts for creating a file are INDEXED, RELATIVE, and SEQUENTIAL.

As discussed previously, you can enter a script directly by specifying the /SCRIPT qualifier on the DCL command line. For example, enter the following command to create an indexed FDL file:

$ EDIT/FDL/CREATE/SCRIPT=INDEXED MY_FDL_FILE

When you select the script, the Edit/FDL utility prompts you for information about the data file. Each prompt consists of a short question, a range of acceptable values (for example, 50-100) or the value type (for example, Keyword, YES/NO, and so forth) in parentheses, and the default answer in brackets. One of the questions in the INDEXED script is shown as follows:

Number of Keys to Define (1-255)[1]     :

In this example, the Edit/FDL utility prompts you for the number of keys you want to define for an indexed data file. The Edit/FDL utility accepts any number from 1 to 255. If you do not specify a value, it assumes that you want to define one key only, the primary key. To accept the default value, press the Return key.

If the Edit/FDL utility requires that you enter a value (that is, no default value is specified for the response), it includes a dash within brackets [-].

When you specify the SEQUENTIAL script or the RELATIVE script, the Edit/FDL utility returns you to the main menu level after finishing the dialog. When you specify the INDEXED script, one of the prompts requests your choice of a design graphics display: a Line_Plot graph or a Surface_Plot graph. After finishing the dialog, the Edit/FDL utility displays the selected graph to help you make your final design choice.

The Line_Plot graph plots bucket size against index depth. All things equal, the size of the buckets determines the number of levels in the index, and the number of levels has a direct effect on the run-time performance of an indexed file. Fewer levels generally reduce the average number of keys searched when the index tree is traversed. However, fewer levels imply more records per data bucket and may cause longer data bucket search times. Thus, the Line_Plot graph helps you decide on the best bucket size for your application. Figure 4.1 shows a Line_Plot graph.

As shown in Figure 4.1, a bucket size of 1 block results in an index with five levels. Increasing the bucket size to 2 blocks reduces the number of index levels to four, but an increase to 5 blocks does not reduce the number of index levels at all. A bucket size of 7 blocks, however, reduces the number of index levels to three.

When you choose the bucket size, remember that the graphs do not display the data level. For example, if you want three levels in the file, then you must limit the number of index levels to two.

The Surface_Plot graphics mode lets you choose a range of values to see their effects. The Edit/FDL utility prompts you to enter a lower and upper bound for one of the following values:

The selected range is displayed along the graph's vertical axis.

The variable on the graph's horizontal axis is bucket size. The numbers in the field portion of the graph show the number of levels at each bucket size for each of the other values.

Figure 4.2 is a Surface_Plot graph that shows a range of values for initial fill factors ranging from 100% to 40%.

The area on the graph within the slash marks represents combinations that RMS finds acceptable. In Figure 4.2, a fill factor of 70% and a bucket size of 10 blocks is the optimum combination. A fill factor of 70% and a bucket size of 15 blocks is a relatively poor combination because it falls outside of the slash boundaries.

If you are sure the information you supplied to the Edit/FDL utility is valid, the best values are those that lie along the left-hand boundary next to the slash marks. If you are not sure that your information is valid, you should choose a value that lies more to the right of the slash boundary.

When you complete the dialog and the Edit/FDL utility presents the graph, you can make changes to certain attributes of the proposed data file. The design is not complete until you specify “FD” for “Finish Design,” at which point the Edit/FDL utility asks a few more questions. You then have the opportunity to return to the main menu to view the file attributes that the Edit/FDL utility has created.

Figure 4.3 shows the attributes that you can alter when the Edit/FDL utility displays the graph. Note that each attribute has a 2-letter mnemonic. To alter an attribute, you specify the corresponding mnemonic. To refresh the display, press the Return key. To begin the final design phase, enter “FD.”

During the final design phase, the Edit/FDL utility gives you an opportunity to supply values for such attributes as TITLE, an optional primary that allows you to label the FDL file. (Most of these questions are also applicable to designing sequential and relative files.) When you have answered the questions, the Edit/FDL utility assigns the values to the FDL attributes and returns you to the main menu level to display the resulting FDL file.

At the main menu, you can select the ADD command to assign values to any attribute the script omitted. Remember that if an attribute does not appear in the FDL file, it assumes the default value. (For a list of the default values for each attribute, see the VSI OpenVMS Record Management Utilities Reference Manual.) To modify an attribute, use the MODIFY command, and to delete an attribute, use the DELETE command.

To create the displayed FDL file, select the EXIT command. To abort the session without creating an FDL file, select the QUIT command.

This section describes what happens when a program does not use a XABITM control block to sense a tag when it opens a file.

When a program opens a DDIF file for record operations and does not sense the tag, RMS assumes that the program wants to access text in the file. In this case, RMS associates the RMS extension with the file access, which provides file attributes that correspond to record-mode access.

When a program opens a DDIF file with the FAB$V_BRO option and does not sense the tag, any subsequent attempt to use block I/O fails. If the program specifies block I/O (FAB$V_BIO) when it invokes the Connect service, the operation fails because the file attributes returned at Open permit record access only. Similarly, if the program specifies the FAB$V_BRO option when it opens the file and then specifies mixed mode (block/record) operations by not specifying RAB$V_BIO at connect time, block operations such as READ and WRITE are disallowed.

RMS does not associate the RMS extension with the file access as part of the Open service if a program opens a DDIF file and senses the stored semantics. This allows the program to specify access semantics with the Connect service. RMS returns the file attributes, including the stored semantics attribute (tag value), to the program as part of the Open service.

When the program subsequently invokes the Connect service, RMS uses the specified operations mode to determine its response. If the program specified FAB$V_BRO with the Open service and then specifies block I/O (RAB$V_BIO) when it invokes the Connect service, RMS does not associate the RMS extension with the file access.

But, if the program specifies record access or FAB$V_BRO when it opens the file and then decides to use record I/O when it invokes the Connect service, RMS compares the access semantics with the file's stored semantics to determine whether to associate the RMS extension with the file access. If the access semantics match the stored semantics, RMS does not associate the RMS extension with the file access. If the access semantics do not match the stored semantics, RMS associates the RMS extension with the file access. In this case, the program must use the Display service to obtain the modified file attributes. If RMS cannot find the appropriate RMS extension, the operation fails and the Connect service returns the EXTNOTFOU error message.

If the application program senses the file's stored semantics, RMS allows mixed-mode operations. In this case, mixed block and record operations are permitted because the application gets record mode file attributes and data from the RMS extension and block mode file attributes and data from the file.

Example 4.7 illustrates a BLISS--32 program that accesses a tagged file from an application program that does not use an RMS extension.

MODULE TYPE$MAIN (
        IDENT = 'X-1',
        MAIN = MAIN,
        ADDRESSING_MODE (EXTERNAL=GENERAL)
        ) =
BEGIN
!
FORWARD ROUTINE
    MAIN : NOVALUE;                        ! Main routine
!
! INCLUDE FILES:
!
LIBRARY 'SYS$LIBRARY:STARLET';
OWN
    NAM            : $NAM(),
    ITEM_BUFF   : BLOCK[ XAB$K_SEMANTICS_MAX_LEN,BYTE ],
    RETLEN,
    FAB_XABITM        :
                $xabitm
                  ( itemlist=
                        $ITMLST_UPLIT
                          ((ITMCOD=XAB$_STORED_SEMANTICS,
                             BUFADR=ITEM_BUFF,
                             BUFSIZ=XAB$K_SEMANTICS_MAX_LEN,
                             RETLEN=RETLEN)),
                    mode = SENSEMODE),
    RAB_ITEMLIST  : BLOCK[ ITM$S_ITEM + 4, BYTE ],
    RAB_XABITM    : $XABITM
                  ( itemlist=RAB_ITEMLIST,
                    mode=SETMODE ),
    FAB         : $FAB( fnm = 'TAGGED-FILE.TEST',
                        nam = NAM,
                        fac = <GET,PUT,UPD>,
                        xab = FAB_XABITM),
    REC             : BLOCK[512,BYTE],
    STATUS,
    RAB         : $RAB( xab = RAB_XABITM,
                        fab = FAB,
                        rsz = 512,
                        rbf = REC,
                        usz = 512,
                        ubf = REC),
    DESC            : BLOCK[8,BYTE] INITIAL(0);
ROUTINE MAIN : NOVALUE =
BEGIN
STATUS = $OPEN( FAB = FAB );
IF NOT .STATUS
THEN
    SIGNAL (.STATUS);
RAB_ITEMLIST[ ITM$W_BUFSIZ ] = .RETLEN;
RAB_ITEMLIST[ ITM$L_BUFADR ] = ITEM_BUFF;
RAB_ITEMLIST[ ITM$W_ITMCOD ] = XAB$_ACCESS_SEMANTICS;
STATUS = $CONNECT( RAB = RAB );
IF NOT .STATUS
THEN
    SIGNAL (.STATUS);
STATUS = $CLOSE( FAB = FAB );
IF NOT .STATUS
THEN
    SIGNAL (.STATUS);
END;
END
ELUDOM

device:[root.][directory-name]filename.type;version

This is the general format of a file specification used to locate a file on the local node, or in an OpenVMS Cluster.

Note that a null node name of the form “::” specifies the local node; this form overrides any default node names.

node::filespec

node “access-control-string”::filespec

The second file specification format includes an access control string. If an access control string is specified or if the process seeking to gain access to a remote file has a proxy login account on the remote node, the specified remote process uses its access rights to locate the file. If an access control string is not specified and a proxy account does not exist on the remote system, the local process may use the default DECnet account, if there is one, to locate the file.

By default the actual password in the access control string is replaced by the word "password" in the expanded or resultant remote file specification for security reasons. A user-mode logical with the dummy "password" string is created by RMS for this purpose; this logical can be used by only one image. The default behavior of not displaying the actual password may be overridden from program control by setting the NAM$V_PWD option in the NAM[L]$L_NOP field. If this option is set, the actual password in the access control string is returned unaltered in the expanded or resultant file specification.

node::“foreign-filespec”

The only action RMS takes with the foreign file specification is to translate the logical node name, if applicable. This format is especially useful when the remote system is not an OpenVMS system and the file specification does not conform to OpenVMS file specification syntax conventions. Refer to the VSI OpenVMS DECnet Networking Manual for more information.

node::“task-spec-string”

For more information about specifying a logical node name or using any of the file specification formats and their associated syntax rules, refer to the VSI OpenVMS User's Manual.

As of OpenVMS V7.2 on Alpha systems, RMS supports a larger character set than was supported in previous versions; it is also larger than the set that is supported under V7.2 on VAX systems. Creating such files is supported only on disks of ODS-5 (or greater) structure level.

          < > : / \ | ? * "

Escape followed by a pair of hexadecimal digits is interpreted as a hexadecimal value for an arbitrary (and otherwise legal) 1-byte character. For example, ^20 represents a space. (The ASCII code for space is hexadecimal 20.)

Escape followed by "_" or by a space represents a space.

Escape followed by "U" followed by 4 hexadecimal digits is interpreted as a 2-byte Unicode character.

    Period (.)?
    Comma (,)
    Semilcolon (;)
    Left bracket ([)
    Right bracket (])
    Percent (%)
    Circumflex (^)
    Amperesand (&)
    Exclamation point (!)
    Pound sign (#)
    Apostrophe (')
    Left parenthesis (()
    Right parenthesis ())
    Plus sign (+)
    Atsign (@)
    Grave accent (`)
    Left brace ({)
    Right brace (})
    Equal sign (=)
    Tilde (~)2

    Period (.)?    Dollar sign ($)
    Minus sign (-)
    Tilde (~)2 

Sequences consisting of the escape character followed by any character not mentioned previously are reserved.

Spaces not preceded by the RMS escape character are removed from the specification by RMS (on Alpha and VAX).

Note that the extended character set applies only to directory names and file names. Device names and node names must conform to ODS-2 requirements.

On ODS-5 disks on Alpha systems, the Extended File Specifications changes added support in OpenVMS Version 7.2 for preserving case (as in uppercase and lowercase letters). If a file is created with lowercase letters from program control, the name, as stored on disk, is lowercase.

From the DCL command interface, file names that are entered at the command prompt with lowercase letters will be translated by default to uppercase before they are passed to RMS. Case may be preserved from the DCL command interface by using the DCL command SET PROCESS/PARSE_STYLE=EXTENDED (also see the SYS$SET_PROCESS_PROPERTIESW system service).

File look-ups, however, are case-blind. For example, the filename "File.Txt" (as stored on an ODS–5 disk) could be accessed with a reference to "FILE.TXT" or "file.txt".

As of OpenVMS Version 7.3-1, an option may be set for file look-ups at either the process or file level to request RMS to either ignore or notice the case sensitivity of file names on ODS–5 disks.

At the process level, the user may request RMS to ignore case by using SET PROCESS/CASE_LOOKUP=BLIND. If a file on an ODS–5 disk already exists whose name matches that of a file being created except for its case, the new file will be created with the same case as the existing file (rather than with the case as entered). This is the default behavior. In contrast, the user may request RMS to notice case by using SET PROCESS/CASE_LOOKUP=SENSITIVE (also see the SYS$SET_PROCESS_PROPERTIESW system service). If the SENSITIVE option is in effect and the user creates more than one file on an ODS–5 disk with the same name differing only in case, each file is treated as a new file.

At the file level, the NAML$V_CASE_LOOKUP flag can be used to instruct RMS to ignore or notice case for a file on an ODS–5 disk (see the NAM$L_INPUT_FLAGS field in the NAML structure in the VSI OpenVMS Record Management Services Reference Manual). NAML$C_CASE_BLIND is set to tell RMS to ignore case or NAML$C_CASE_LOOKUP_SENSITIVE to notice case when creating, deleting or searching for a file on an ODS–5 disk. If the NAML structure is not used or this flag is zero, the current process setting for CASE_LOOKUP is used.

The SET PROCESS/PARSE_STYLE qualifier is independent of the /CASE_LOOKUP qualifier. If the creation, deletion, or search of files on an ODS–5 disk is being done using the DCL command interface and case is relevant, /PARSE_STYLE=EXTENDED must be used to inform the DCL interface to preserve the case specified in the DCL command. The /CASE_LOOKUP qualifier instructs RMS whether to ignore or notice the case (either preserved or not).

On Alpha systems, a system service, SYS$CVT_FILENAME, is available to convert names between their RMS-interface form and their ACP-XQP on-disk form. See the VSI OpenVMS System Services Reference Manual: A-GETUAI for details on its use.

Using the tilde (~) as the first character in a file name in the VSI C Run-Time Library (CRTL) allows a programmer to specify both UNIX style and OpenVMS style file specifications to routines such as creat() and fopen().

In UNIX file specifications, a tilde (~) in the first character of a pathname represents the user's home directory. However, in an OpenVMS extended file name, a tilde is legal anywhere in a file name or directory name.

To preserve backward compatibility, the CRTL will continue to interpret a leading tilde (~) to mean the user's home directory. To pass an OpenVMS file name that begins with a tilde (~) to a CRTL routine that accepts UNIX style file specifications, specify the tilde preceded by the escape character (^). For example, ^~.

This section describes the special processing needed to implement sticky defaults when a search list is used in a related file specification for an input file parse. The term sticky default means that file specification components from the first file specification are applied as defaults to the next file specification component, eliminating the need, for instance, to specify the device specification for each file specification when all the files are located on the same device.

The related file specification provides defaults when a related file name block is present. To use the related file specification, the file access block must specify the address of the primary file's name block (in the FAB$L_NAM or FAB$L_NAML field), and that name block must specify the address of the related file's name block (in the NAM$L_RLF or NAML$L_RLF field). The related file's name block must specify the address of a valid file specification in the resultant string (NAM$L_RSA/NAM$B_RSS or NAML$L_LONG_RESULT/NAML$L_LONG_RESULT_ALLOC) field. Typically, an RMS file service (other than Parse) places the file specification in the resultant string.

You can specify whether the related file is used as an input file specification or an output file specification by setting (output file specification parsing) or resetting (input file specification parsing) the output-file parse (FAB$V_OFP) bit in the file-processing options (FAB$L_FOP) field.

When an input file specification is being parsed, you can have multiple related name blocks by specifying the second related file's name block address in the NAM$L_RLF or NAML$L_RLF field of the first related name block, the address of the third related name block in the NAM$L_RLF or NAML$L_RLF field of the second name block, and so forth. The use of multiple related name blocks is especially useful for search lists; one related name block might contain a file type that can be used by any file specification in a search list, another might contain the full file specification that was produced by the first search list file specification, and another might contain the full file specification produced by the second search list file specification. This method allows the file specifications in a search list to provide sticky defaults, a characteristic associated with DCL command lines that contain multiple file specifications.

$ DEFINE WORK DISK1:[MINE],DISK2:[GROUP]

To process lists of input files—such as WORK:A,B,C,—your program must supply the string WORK:A as the related file specification, not DISK2:[GROUP]A.DAT. The routines LIB$FIND_FILE and LIB$FILE_SCAN are provided to perform this special processing; consult the VSI OpenVMS RTL Library (LIB$) Manual for additional information; also refer to Example 5.2, which shows how to call the LIB$FIND_FILE routine.

When the FAB$L_FOP field FAB$V_OFP bit is set and a node name is not present, VAX RMS processes the related file specification as an output file specification as shown below.

When the FAB$V_OFP bit is set and a node name is present, RMS processes the related file specification as an output file specification, as shown in the following table:

As shown in the previous table, a wildcard character in an output directory specification is subject to the following syntax restrictions:

RMS processes wildcard characters in an output directory specification as follows: