You can mount a maximum of 500 disks in two- or three-member shadow sets on a standalone system or in an OpenVMS Cluster system. A limit of 10,000 single member shadow sets is allowed on a standalone system or on an OpenVMS Cluster system. Dismounted shadow sets, unused shadow sets, and shadow sets with no write bitmaps allocated to them are included in this total. These limits are independent of controller and disk type. The shadow sets can be mounted as public or private volumes.

Starting with OpenVMS Version 7.3, the SHADOW_MAX_UNIT system parameter is available for specifying the maximum number of shadow sets that can exist on a node. For more information about SHADOW_MAX_UNIT, see Section 3.3.1.

OpenVMS Version 8.4 supports six-member shadow sets as compared to the previous three-member shadow sets. This is useful for multisite disaster tolerant configuration. In a three-member shadow set, a three-site disaster tolerant configuration has only one shadow member per site. In this scenario, when two sites fail, the member left out in the surviving site becomes a single point of failure. With six-member shadow set support, you can have two members of a shadow set in each of the three sites providing high availability.

For example:

DS10 $ SHOW DEV DSA5678:
Device                Device          Error    Volume     Free   Trans  Mnt
 Name                 Status          Count    Label     Blocks  Count  Cnt
DSA5678:              Mounted             0  SIXMEMBER   682944      1    1
$6$DKB0:    (WSC236)  ShadowSetMember     0  (member of DSA5678:)
$6$DKB100:  (WSC236)  ShadowSetMember     0  (member of DSA5678:)
$6$DKB200:  (WSC236)  ShadowSetMember     0  (member of DSA5678:)
$6$DKB300:  (WSC236)  ShadowSetMember     0  (member of DSA5678:)
$6$DKB400:  (WSC236)  ShadowSetMember     0  (member of DSA5678:)
$6$DKB500:  (WSC236)  ShadowSetMember     0  (member of DSA5678:)
DS10 $ 

You can shadow system disks as well as data disks. Thus, a system disk need not be a single point of failure for any system that boots from that disk. System disk shadowing becomes especially important for OpenVMS Cluster systems that use a common system disk from which multiple computers boot. Volume shadowing makes use of the OpenVMS distributed lock manager, and the quorum disk must be accessed before locking is enabled. Note that you cannot shadow quorum disks.

Integrity server, Alpha, and x86 systems can share data on shadowed data disks, but separate system disks are required — one for each architecture.

On each Integrity server and x86 system disk, there can exist up to two File Allocation Table (FAT)s partitions that contain OpenVMS boot loaders, Extensible Firmware Interface (EFI) applications and hardware diagnostics. The OpenVMS bootstrap partition and, when present, the diagnostics partition are respectively mapped to the following container files on the OpenVMS system disk:

SYS$LOADABLE_IMAGES:SYS$EFI.SYS
SYS$MAINTENANCE:SYS$DIAGNOSTICS.SYS

The contents of the FAT partitions appear as fsn: devices at the console EFI Shell> prompt. The fsn: devices can be directly modified by the user command input at EFI Shell> prompt and by the EFI console or EFI diagnostic applications. Neither OpenVMS nor any EFI console environments that might share the system disk are notified of partition modifications; OpenVMS and console environments are unaware of console modifications. You must ensure the proper

coordination and proper synchronization of the changes with OpenVMS and with any other EFI consoles that might be in use.

You must take precautions when modifying the console in configurations using either or both of the following:

You must preemptively reduce the OpenVMS system disk environments to a single-member host-based volume shadow set or to a non-shadowed system disk, and you must externally coordinate access to avoid parallel accesses to the Shell> prompt whenever making shell-level modifications to the fsn: devices, such as:

If you do not take these precautions, any modifications made within the fsn: device associated with the boot partition or the device associated with the diagnostic partition can be overwritten and lost immediately or after the next OpenVMS host-based volume shadowing full-merge operation.

For example, when the system disk is shadowed and changes are made by the EFI console shell to the contents of these container files on one of the physical members, the volume shadowing software is unaware that a write is done to a physical device. If the system disk is a multiple member shadow set, you must make the same changes to all of the other physical devices that are the current shadow set members. If this is not done, when a full merge operation is next performed on that system disk, the contents of these files might regress. The merge operation might occur many days or weeks after any EFI changes are made.

Furthermore, if a full merge is active on the shadowed system disk, you must not make changes to either file using the console EFI shell.

To suspend a full merge that is in progress or to determine the membership of a shadow set, see Chapter 8.

The precautions are applicable only for the Integrity server and x86 system disks that are configured for host-based volume shadowing, or are configured and shared across multiple OpenVMS Integrity server and OpenVMS x86 systems. Configurations that are using controller-based RAID, that are not using host-based shadowing with the system disk, or that are not shared with other OpenVMS Integrity server or OpenVMS x86 systems, are not affected.

To use the minicopy feature in a mixed-version OpenVMS Cluster system, every node in the cluster must use a version of OpenVMS that supports this feature.

Shadow sets also can be constituents of a bound volume set or a stripe set. A bound volume set consists of one or more disk volumes that have been bound into a volume set by specifying the /BIND qualifier with the MOUNT command. Section 1.5 describes shadowing across OpenVMS Cluster systems. Chapter 10 contains more information about striping and how RAID (redundant arrays of independent disks) technology relates to volume shadowing.

This section provides guidelines for using volume shadowing parameters.

ALLOCLASS

The ALLOCLASS parameter is used to specify an allocation class that forms part of a device name. The purpose of allocation classes is to provide unique and unchanging device names. When using Volume Shadowing for OpenVMS on a single system or on an OpenVMS Cluster system, a nonzero allocation class value is required for each physical device in the shadow set. For more information about using allocation classes, see the VSI OpenVMS Cluster Systems Manual.

SHADOWING

The SHADOWING parameter enables or disables volume shadowing on your system, as shown in Table 3.2.

SHADOW_HBMM_RTC

SHADOW_HBMM_RTC is used to specify, in seconds, how frequently each shadow set on this system has its modified block count compared with its reset threshold. If the modified block count exceeds the reset threshold, the bitmap for that shadow set is zeroed. This comparison is performed for all shadow sets mounted on the system that have HBMM bitmaps. The reset threshold is specified by the RESET_THRESHOLD keyword in the /POLICY qualifier of the SET SHADOW command. When the comparison is made, the modified block count might exceed the reset threshold by a small increment or by a much larger amount. The difference depends on the write activity to the volume and the setting of this parameter.

The default setting of SHADOW_HBMM_RTC is 150 seconds.

You can view the reset threshold setting and the modified block count, since the last reset, in the SHOW SHADOW command display. For guidelines on setting the reset threshold values and a sample SHOW SHADOW display, see Section 8.5.2. For a SHOW SHADOW display that includes a modified block count greater than the reset threshold value, refer to the VSI OpenVMS DCL Dictionary: N-Z.

SHADOW_MAX_COPY

The SHADOW_MAX_COPY parameter controls how many parallel copy and merge operations are allowed on a given node. (Copy and merge operations are described in Chapter 6.) This parameter provides a way to limit the number of copy and merge operations in progress at any time.

The value of SHADOW_MAX_COPY can range from 0 to 200. The default value is specific to the OpenVMS version. You can determine the default value by looking at the parameter setting. When the value of the SHADOW_MAX_COPY parameter is 4, and you mount five multivolume shadow sets that all need a copy operation, only four copy operations can proceed. The fifth copy operation must wait until one of the first four copies completes.

For example, the default value of 4 may be too high for a small node. (In particular, satellite nodes should have SHADOW_MAX_COPY set to a value of 0.) Too low a value for SHADOW_MAX_COPY unnecessarily restricts the number of operations your system can effectively handle and extends the amount of time it takes to merge all of the shadow sets.

SHADOW_MAX_COPY is a dynamic parameter. Changes to the parameter affect only future copy and merge operations; current operations (pending or already in progress) are not affected.

SHADOW_MAX_UNIT

The SHADOW_MAX_UNIT specifies the number of shadow sets that can exist on a node and determines the memory reserved for the bitmap for each shadow set. (See Section 1.3.1.) The important thing to note about this value is that any shadow set that has been created, regardless of whether it is in use, is included in this total. Because this is not a dynamic system parameter, you must be very careful when determining the value to use. If you have to change this parameter, you must reboot the system.

The default value for OpenVMS Alpha systems is 500.

Note that this parameter does not affect the naming of shadow sets. For example, with the default value of 100, a device name such as DSA999 is still valid.

SHADOW_MBR_TMO

The SHADOW_MBR_TMO parameter controls the amount of time the system tries to fail over physical members of a shadow set before removing them from the set. SHADOW_MBR_TMO is a dynamic parameter that you can change on a running system.

If you specify zero, a default delay is used. The default delay is specific to the version of OpenVMS running on your system. For shadow sets in an OpenVMS Cluster configuration, the value of SHADOW_MBR_TMO must be set to the same value on each node.

Determining the correct value for SHADOW_MBR_TMO is a trade-off between rapid recovery and high availability. If rapid recovery is required, set SHADOW_MBR_TMO to a low value. This ensures that failing shadow set members are removed from the shadow set quickly and that user access to the shadow set continues. However, removal of shadow set members reduces data availability and, after the failed member is repaired, a full copy operation is required when it is mounted back into the shadow set.

If high availability is paramount, set SHADOW_MBR_TMO to a high value. This allows the shadowing software additional time to regain access to failed members. However, user access to the shadow set is stalled during the recovery process. If recovery is successful, access to the shadow set continues without the need for a full copy operation, and data availability is not degraded. Setting SHADOW_MBR_TMO to a high value may be appropriate when shadow set members are configured across LANs that require lengthy bridge recovery time.

Shadowing uses a timer to adhere to the number of seconds specified by the SHADOW_MBR_TMO parameter. For directly connected SCSI devices that have been powered down or do not answer to polling, the elapsed time before a device is removed from a shadow set can take several minutes.

To modify SHADOW_MBR_TMO for an existing shadow set member, see the SET SHADOW/RECOVERY_OPTIONS=DELAY_PER_SERVED_MEMBER=n command, which is described in Section 4.8.

SHADOW_PSM_DLY

SHADOW_PSM_DLY allows the system manager to adjust the delay that Shadowing adds automatically when a copy or merge operation is needed on a shadow set that is mounted on many systems.

The Shadowing facility attempts to perform the operation on a system that has a local connection to all the shadow set members. Shadowing implements the copy or merge operation by adding a time delay based on the number of shadow set members that are MSCP-served to the system. No delay is added for local members. Therefore, a system with all locally accessible shadow set members usually performs the copy or merge before a system on which one or more members is served and is therefore delayed.

When a shadow set is mounted on a system, the value of SHADOW_PSM_DLY is used as the default shadow set member recovery delay for that shadow set. To modify SHADOW_PSM_DLY for an existing shadow set, see the SET SHADOW/RECOVERY_OPTIONS= DELAY_PER_SERVED_MEMBER=n command, which is described in Section 4.8.

SHADOW_PSM_DLY is a static parameter; its range is 0 to 65535 seconds. The default value is 30 seconds for each MSCP served shadow set member.

SHADOW_REC_DLY

SHADOW_REC_DLY governs the system behavior after a system failure or after a shadow set is aborted. The value of the SHADOW_REC_DLY parameter is added to the value of the RECNXINTERVAL parameter to determine how long a system waits before it attempts to manage a merge or copy operation on any shadow sets that it has mounted.

SHADOW_REC_DLY can be used to predict the systems that can perform recovery operations in an OpenVMS Cluster. This is done by setting lower values of SHADOW_REC_DLY on systems that are preferred to handle recovery operations and higher values of SHADOW_REC_DLY on the remaining systems.

SHADOW_REC_DLY is a dynamic parameter; its range is 0 to 65535 seconds. The default value is 20 seconds.

For more information about controlling which systems perform the merge or copy operations, see Section 4.9.5.

SHADOW_SYS_DISK

A SHADOW_SYS_DISK parameter value of 1 enables shadowing of the system disk. A value of 0 disables shadowing of the system disk. A value of 4097 enables a minimerge. The default value is 0.

If you enable a minimerge of the system disk, you must also configure your system to write a dump to a non-shadowed, non-system disk of your choice. This is known as dump off system disk (DOSD). For more information on DOSD, see the VSI OpenVMS System Manager's Manual, Volume 2: Tuning, Monitoring, and Complex Systems.

In addition, you must specify a system-disk, shadow-set virtual unit number with the SHADOW_SYS_UNIT system parameter, unless the desired system disk virtual unit number is DSA0.

SHADOW_SYS_TMO

You can use the SHADOW_SYS_TMO parameter in two ways: during the booting process and during normal operations. SHADOW_SYS_TMO is a dynamic parameter that you can change on a running system.

During the booting process, you can use this parameter on the first node in the cluster to boot and to create a specific shadow set. If the proposed shadow set is not currently mounted in the cluster, use this parameter to extend the time a booting system waits for all former members of the system disk shadow set to become available.

The second use of this parameter comes into effect once the system successfully mounts the shadow set and begins normal operations. Just as the SHADOW_MBR_TMO parameter controls the time the operating system waits for failing members of an application disk shadow set to rejoin the shadow set, the SHADOW_SYS_TMO parameter controls how long the operating system waits for failing members of a system disk shadow set. All nodes using a particular system disk shadow set must have their SHADOW_SYS_TMO parameter equal to the same value, after normal operations begin. Therefore, after booting, this parameter applies only to members of the system disk shadow set.

The default value is OpenVMS version specific. You can set a range of up to 65,535 seconds if you want the system to wait longer than the default for all members to join the shadow set.

SHADOW_SYS_UNIT

The SHADOW_SYS_UNIT parameter, which must be used when the SHADOW_SYS_DISK parameter is set to 1, contains the virtual unit number of the system disk.

The SHADOW_SYS_UNIT parameter is an integer value that contains the virtual unit number of the system disk. The default value is 0. The maximum value allowed is 9999. This parameter is effective only when the SHADOW_SYS_DISK parameter has a value of 1. This parameter must be set to the same value on all nodes that boot off a particular system disk shadow set. SHADOW_SYS_UNIT is not a dynamic parameter.

SHADOW_SYS_WAIT

Use the SHADOW_SYS_WAIT parameter to extend the time a booting system waits for all current members of a mounted system disk shadow set to become available to this node. SHADOW_SYS_WAIT is a dynamic parameter that you can change on a running system (for debugging purposes only). The shadow set must already be mounted by at least one other cluster node for this parameter to take effect. The default value is 256 seconds. Change this parameter to a higher value if you want the system to wait more than the 256-second default for all members to join the shadow set. This parameter has a range of 1 through 65,535 seconds.

   .
   .
   .
! Volume Shadowing Parameters:
SHADOWING=2             ! Enables phase II shadowing

SHADOW_SYS_DISK=1       ! Enables system disk shadowing

SHADOW_SYS_UNIT=7       ! Specifies 7 as the virtual unit number
                          of the system disk

SHADOW_MAX_COPY=4       ! Specifies that 4 parallel copies can occur at one time

SHADOW_MBR_TMO=120      ! Allows 120 seconds for physical members to fail over
                        ! before removal from the shadow set

   .
   .
   .

See the VSI OpenVMS System Manager's Manual, Volume 2: Tuning, Monitoring, and Complex Systems for complete information about invoking AUTOGEN and specifying the appropriate command qualifiers to perform the desired AUTOGEN operations.

It is sometimes useful to use the SYSGEN command SHOW to display the values of system parameters.

$ MCR SYSGEN
SYSGEN> SHOW SHADOWING
Parameter Name   Current  Default  Minimum  Maximum  Unit        Dynamic
--------------   -------  -------  -------  -------  ----        -------
SHADOWING              2        0        0        2  Coded-value
SYSGEN>

You can use the INITIALIZE/SIZE command to create a file system that is smaller than the current physical size of the volume. If you have a 36-GB disk and you anticipate adding an 18-GB disk in the future, then you can initialize the disk with the following command:

$ INIT/SIZE=36000000 $1$DGAnlabel

In this example, 18GB disk = 36,000,000 blocks of 512 bytes each approximately.

If you are adding a new volume to your system, increase the expansion limit on the volume when you initialize the disk with INITIALIZE/LIMIT. To increase the expansion limit on volumes already in use, plan to increase the expansion limit during the next convenient maintenance period using the command SET VOLUME/LIMIT.

If INITIALIZE/LIMIT is used, the default cluster size of the /CLUSTER_SIZE qualifier is 8. This value controls how much space the bitmap occupies. You can later expand the volume (using the SET VOLUME volume-name/SIZE=x command) while the device is still mounted if your storage requirements grow unexpectedly.

VSI strongly recommends that you use the /ERASE qualifier. By using the /ERASE qualifier, a subsequent merge operation will be substantially reduced.

If you omit the /ERASE qualifier, then the portions of the volume that do not contain file system data structures contain indeterminate data. This data can differ from one shadow set member to another. Make sure to take this into account when using utilities that compare all of the LBNs between shadow set members.

The next time a full merge operation occurs, the presence of this indeterminate data causes the merge to take much longer than it takes without the use of the INITIALIZE/SHADOW/ERASE command. When this full merge completes, the LBNs contain identical data, and the storage control block (SCB) no longer indicates that the /ERASE qualifier was omitted from the INITIALIZE/SHADOW command.

Note, however, that a side effect of using /ERASE is that the ERASE volume attribute is set. In effect, each file on the volume is erased when it is deleted. Another side effect is that an INITIALIZE/ERASE operation is always slower than an INITIALIZE/NOERASE operation. The disks are erased sequentially, which effectively doubles or triples the time it takes for the command to complete. If the disks are large, consider performing multiple, simultaneous INITIALIZE/ERASE commands (with the /SHADOW qualifier) to erase the disks. After all the commands have completed, then perform an INITIALIZE/SHADOW command with the /ERASE qualifier.

You can remove the ERASE volume attribute by issuing the SET VOLUME/NOERASE_ON_DELETE command.

For more information about these DCL commands and qualifiers, see the VSI OpenVMS DCL Dictionary: N-Z.

Shadow set members can differ in size, that is, they can have different nonzero values for Total Blocks. If devices of different sizes are specified in the INITIALIZE command, and /SIZE or /LIMIT or both are omitted, the default values for these qualifiers take effect. The default value for /SIZE (for the logical volume size for the device) is the smallest member’s MAXBLOCK value. The default value for /LIMIT (for future expansion) is the largest member’s MAXBLOCK value, which is used to compute the expansion limit.

You can view the Total Blocks value by entering the SHOW DEVICE/FULL command. If a device has never been mounted or initialized on this system, the SHOW DEVICE/FULL command for the device does not display a value for Total Blocks. To correct this condition, either mount and then dismount the device, or initialize the device. The Total Blocks value is then displayed by SHOW DEVICE/FULL.

The use of INITIALIZE/SHADOW requires the VOLPRO privilege.

Note that the INITIALIZE/SHADOW command must not be used to initialize a disk to be added to an existing shadow set, since there is no benefit to be gained.

INITIALIZE/SHADOW=(device_name1, device_name2, device_name3) label

You may occasionally find it useful to specify the /NOASSIST qualifier on the MOUNT command. For example, you can use the MOUNT/NOASSIST command in startup files to avoid failure of a MOUNT command when a device you specify in the command is not available. The /NOASSIST qualifier can be used in startup files because operator intervention is impossible during startup.

$ MOUNT/SYS DSA65:/SHADOW=($4$DIA6,$4$DIA5) GALEXY/NOASSIST
%MOUNT-I-MOUNTED, GALEXY mounted on _DSA65:
%MOUNT-I-SHDWMEMSUCC, _$4$DIA6: (READY) is now a valid member of the shadowset
%MOUNT-I-SHDWMEMFAIL, $4$DIA5 failed as a member of the shadow set
-SYSTEM-F-VOLINV, volume is not software enabled

Even though device $4$DIA5 is not available for mounting, the MOUNT command continues to create the shadow set with $4$DIA6 as its only member. If the command did not include the /NOASSIST qualifier, the MOUNT command would not mount the shadow set.

When you create a shadow set, you must specify either the /SYSTEM qualifier or the /CLUSTER qualifier, or both (see Table 4.2) to provide access for all users on a single system or on a cluster.

In Example 4.3, if the shadow set (identified by its virtual unit name DSA2) is not currently mounted, the first command creates a shadow set with one shadow set member; the second command adds two more members to the same shadow set. An automatic copy operation causes any data on the second and third volumes to be overwritten as the shadow set members are added.

$ MOUNT DSA2: /CLUSTER /SHADOW=$6$DIA4: PEAKSISLAND DISK$PEAKSISLAND
$ MOUNT DSA2: /SYSTEM/SHADOW=($6$DIA5:,$6$DIA6:) PEAKSISLAND DISK$PEAKSISLAND

$  MOUNT/CONFIRM/SYSTEM DSA23: /SHADOW=($4$DUA9,$4$DUA3) volume-label

The command specifies both the currently active shadow set member ($4$DUA9) and the new member ($4$DUA3). Although it is not necessary to include them when mounting additional physical devices, you can specify current shadow set members without affecting their membership state.

Note that when you add volumes to an existing shadow set mounted across an OpenVMS Cluster system, the shadowing software automatically adds the new members on each OpenVMS Cluster node.

$ MOUNT/SYSTEM DSA4: /SHADOW = ($3$DIA7:, $3$DIA8:)
FORMERSELF
%MOUNT-I-MOUNTED, FORMERSELF   mounted on DSA4:
%MOUNT-I-SHDWMEMSUCC, _$3$DIA7: (DISK300) is now a valid member of
                      the shadow set
%MOUNT-I-SHDWMEMSUCC, _$3$DIA8: (DISK301) is now a valid member of
                      the shadow set

$ MOUNT/SYSTEM DSA4: /SHADOW = $3$DIA6:  FORMERSELF

%MOUNT-I-SHDWMEMCOPY, _$3$DIA6: (DISK302) added to the shadow set
                      with a copy operation

In this example, the first command creates a shadow set whose virtual unit name is DSA4. The member disks are $3$DIA7 and $3$DIA8. The second command mounts the disk $3$DIA6 and adds it to shadow set DSA4. The shadow set now includes three members: $3$DIA6, $3$DIA7, and $3$DIA8. In this example, when you add $3$DIA6 after the shadow set already exists, the added volume becomes the target of a copy operation.

When you specify more than one disk, the shadowing software automatically determines the correct copy operation to perform in order to make shadow set members consistent with each other (see Section 6.2 for details). The Mount utility interprets information recorded on each member to determine whether a member requires a copy operation, a merge operation, or no copy operation. If you are not sure which disks might be targets of copy operations, you can specify the /CONFIRM qualifier or the /NOCOPY qualifier as a precaution against overwriting important data when you mount a disk. With the /NOCOPY qualifier, you disable the copy operation.

$ MOUNT/NOCOPY DSA2: /SHADOW=($1$DUA4:,$1$DUA6:,$1$DUA7:) -
_$ SHADOWVOL DISK$SHADOWVOL 
%MOUNT-F-SHDWCOPYREQ, shadow copy required
%MOUNT-I-SHDWMEMFAIL, DUA7: failed as a member of the shadow set
%MOUNT-F-SHDWCOPYREQ, shadow copy required 
$ MOUNT/COPY DSA2: /SHADOW=($1$DUA4:,$1$DUA6:,$1$DUA7:) -
_$ SHADOWVOL DISK$SHADOWVOL
%MOUNT-I-MOUNTED, SHADOWVOL    mounted on _DSA2:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA4: (VOLUME001) is now a valid member of
                      the shadow set
%MOUNT-I-SHDWMEMSUCC, _$1$DUA6: (VOLUME002) is now a valid member of
                      the shadow set
%MOUNT-I-SHDWMEMCOPY, _$1$DUA7: (VOLUME003) added to the shadow set
                      with a copy operation 

Occasionally, you have to mount a physical shadow set member as a nonshadowed disk. By default, when a shadow set member is mounted outside a shadow set, the Mount utility automatically write-locks the disk. This provides a safeguard against accidental modification, thereby allowing the disk to be remounted into a shadow set at a later time.

To override this default behavior, include the /OVERRIDE=SHADOW_MEMBERSHIP qualifier on the MOUNT command as shown in the following example:

$ MOUNT/OVERRIDE=SHADOW_MEMBERSHIP $4$DUA20: WORKDISK

This command ignores shadow set membership status and mounts a former shadow set member on $4$DUA20 as a nonshadowed disk with write access.

Figure 4.1 shows a shadow set DSA42, whose members are devices $1$DGA1000 and $1$DGA2000. Systems at Site A or Site B have direct access to all devices at both sites via Fibre Channel connections. XYZZY is a theoretical point between the two sites. If the Fibre Channel connection were to break at this point, each site could access different “local” members of DSA42 without error.

For the purpose of this example, Site A will be the sole site chosen to retain access to the shadow set.

The /DEMAND_MERGE qualifier was created to force a merge operation on shadow sets that were created with the INITIALIZE/SHADOW command without specifying the /ERASE qualifier. The /DEMAND_MERGE qualifier ensures that all blocks not in use by active files are the same. The system manager can enter this command at a convenient time. If the /ERASE qualifier was not used when the shadow set was created with /INITIALIZE/SHADOW, and the SET SHADOW/DEMAND_MERGE command has not been executed, then the higher overhead of a full merge operation on this shadow set is encountered after a system failure.

System managers can also use the SET SHADOW/DEMAND_ MERGE command if the ANALYZE/DISK/SHADOW command found differences between the members of the shadow set (see Section 4.11.4).

The SHOW SHADOW command reports on the status of the specified shadow set and indicates whether a merge or copy operation is required, depending on the qualifier that you specify. If a merge or copy operation is required, this command reports whether it is pending or in progress. The qualifiers are described in this section. To use this command, specify the shadow set’s virtual unit name, followed by the qualifiers you want to use, as shown in the following example:

$ SHOW SHADOW DSAnnnn:/qualifier/qualifier/

/ACTIVE

This qualifier returns one of three possible states:

/COPY

This qualifier returns one of three possible states:

/MERGE

This qualifier returns one of three possible states:

/OUTPUT=file-name

This qualifier outputs any messages to the specified file.

Example 4.8 shows sample output from the SHOW SHADOW command:

$SHOW SHADOW DSA716:
_DSA716: TST716
  Virtual Unit SCB Status: 0001 - normal
  Local Virtual Unit Status: 00000010 - Local Read

  Total Devices          2    VU_UCB          810419C0
  Source Members         2    SCB LBN         000009C8
  Act Copy Target        0    Generation      00A15F90
  Act Merge Target       0    Number          EDA9D786
  Last Read Index        0    VU Site Value          5
  Master Mbr Index       0    VU Timeout Value    3600
  Copy Hotblocks         0    Copy Collisions        0
  SCP Merge Repair Cnt   0    APP Merge Repair Cnt   0


  Device  $252$DUA716             Master Member
  Index   0   Status   000000A0   src,valid
  Ext.   Member Status       00
  Read Cost                  42   Site  5
  Member Timeout             120  UCB   8116FF80

  Device $252$DUA1010
  Index   1   Status   000000A0   src,valid
  Ext. Member Status         00
  Read Cost                 500   Site  3
  Member Timeout            120   UCB   811DD500

If a system fails or if it aborts a shadow set, most commonly through mount verification, it is termed as a significant event. When one of these significant events occurs, all systems in the cluster are notified automatically. This notification causes all shadow server processes to stop any full merge or full copy operations and release all the resources performing these operations. Thus, every system can reallocate its resources to newer, higher-priority work.

After a predetermined delay, each system with a non-zero SHADOW_MAX_COPY setting begins to process the shadow sets that are in a transient state, according to their priority. The predetermined delay is governed by the new system parameter SHADOW_REC_DLY. (For more information about SHADOW_REC_DLY, see Table 3.1 and Section 4.9.5.) Every system allocates the available SHADOW_MAX_COPY resources based on a shadow set's priority.

A shadow set is in a steady state when it is known that all members contain identical data. If a shadow set has one or more of the following operations pending, or one operation active, it is said to be in a transient state:

While a combination of these transient states is valid, only one operation at a time can be performed. For example, consider that HBMM is not enabled. After a device is added to a shadow set, it is marked as being in a full copy transient state. If the system on which this shadow set is mounted fails, the shadow set is further marked as being in a full merge state. In this example, the full copy operation is performed before the full merge is started.

Shadow set operations for a specific shadow set are performed in the following order:

When first mounted on a system, every shadow set is assigned a default priority of 5000. You can assign a unique priority to every mounted shadow set on a per-system basis using the SET SHADOW/PRIORITY=n DSAn command. Every shadow set can have a unique priority per system, or shadow sets can be assigned the same priority. Shadow sets with the same priority are managed in a consistent way for each release. However, the order in which shadow sets with the same priority are managed may change from release to release because of changes to the algorithm. Therefore, if the order is important, assign them different priorities.

The valid range for priority values is 0 through 10,000. The higher the assigned value, the higher the priority. To ensure that high-priority volumes are merged (or copied) before less important volumes, use SET SHADOW/PRIORITY=n DSAn command to override the default priority assignment on a system.

A priority level of 0 has a unique meaning. It means that the shadow set is not considered for merge or copy operations on this system.

You can display the priority of a shadow set on a specific system by issuing the following command:

$ SHOW SHADOW/BY_PRIORITY DSAn:

This command displays the current priority and status of the specified shadow set. If any copy or merge operations are in progress, the node on which the operation is progressing is displayed, along with its progress. For example:

$ SHOW SHADOW DSA1104/BY_PRIORITY
Device     Mbr                                        Active
 Name      Cnt   Priority   Virtual   Unit   State   on Node
_DSA1104:   2      5000     Merge    Active  (29%)     MAX

You can use the SHOW SHADOW/BY_PRIORITY command to display the priority level and the status for all of the shadow sets that exist on the system. The status indicates whether the shadow set is currently undergoing a copy or merge operation or whether one is required. If either or both operations are underway, the systems on which they occur are identified in the display, as shown in the following example:

$ SHOW SHADOW/BY_PRIORITY

Device     Mbr                                                    Active
 Name      Cnt   Priority   Virtual   Unit   State                on Node
_DSA106:    2      10000    Steady    State
_DSA108:    3       8000    Steady    State
_DSA110:    3       8000    Steady    State
_DSA112:    3       8000    Steady    State
_DSA114:    1       7000    Steady    State
_DSA116:    1       7000    Steady    State
_DSA150:    2       7000    Steady    State
_DSA152:            7000    Not Mounted on this node
_DSA154:    3       6000    Steady    State
_DSA156:    1       6000    Steady    State
_DSA159:    2       5000    Steady    State
_DSA74:     3       5000    Merge    Active (47%)                   CASSID
_DSA304:   2+1      5000    Merge    Active (30%), Copy Active (3%) MAX
_DSA1104:   2       5000    Merge    Active (29%)                   MAX
_DSA300:   2+1      5000    Merge    Active (59%), Copy Active (0%) MAX
_DSA0:     1+2      5000    Copy     Active (83%)                   CASSID
_DSA3:      2       3000    Steady    State
_DSA100:    2       3000    Steady    State
_DSA102:    1       3000    Steady    State
_DSA104:    3       3000    Steady    State
Total of 19 Operational shadow sets; 0 in Mount Verification; 1 not mounted
$

In this example, the 20 shadow sets on this system are displayed in their priority order. In the event of a failure of another system in the cluster that has these shadow sets mounted, the shadow sets are merged in this order on the system.

The Mbr Cnt field shows the number of source members in each shadow set. If members are being added via a copy operation, this is indicated by +1 or +2. Therefore, 2+1 indicates two source members and one member being added. The notation 1+2 indicates one source member and two members being copied into the set.

The summary line provides the total number of shadow sets that were found to be in the various conditions. “Operational shadow sets” are shadow sets that are mounted with one or more members and may or may not have copy or merge operations occurring. These shadow sets are available to applications for reads and writes. “Mount Verification” indicates the number of shadow sets that are in some mount verification state. Shadow sets that have exceeded their mount verification timeout times are also included in this total.

For additional examples, see the SHOW SHADOW examples in the VSI OpenVMS DCL Dictionary: N-Z.

When a system fails or aborts a shadow set, this significant event causes every shadow set to be reassessed by all other systems with that shadow set mounted. All active minimerge, full merge, or copy operations cease at this time, returning their resources to those systems. (However, if a system is performing a minicopy operation, that operation continues to completion.)

These systems wait a predetermined amount of time, measured in seconds, before each attempts to manage any shadow set in a transient state. This pause is called a significant-event recovery delay. It is the total of the values specified for two system parameters, SHADOW_REC_DLY and RECNXINTERVAL. (The default value for each is 20 seconds.)

If the value of the significant-event recovery delay is the same on all systems, it is not possible to predict which systems manage which shadow set. However, by making the value of the significant-event recovery delay different on all systems, you can predict when a specific system begins to manage transient-state operations.

A merge transient state is an event that cannot be predicted. The management of merge activity, on a specific system for multiple shadow sets, can be predicted if the priority level settings for the shadow sets differ.

The following example illustrates how the priority level is used to select shadow sets when only merge operations are involved. In this example:

$ SET SHADOW/PRIORITY=7000 DSA1:
$ SET SHADOW/PRIORITY=3000 DSA42:
! DSA20: and DSA22: are at the default priority level of 5000

When one of the systems in this example fails, all shadow sets are put into a merge-required state. After the significant-event recovery delay time elapses, this system evaluates the shadow sets, and the operations are performed in the following order:

A copy transient state can be predicted by the user because it is the result of direct user action. Therefore, a full copy operation caused by adding a device to a shadow set is not considered a significant event in the cluster. The copy operation is managed by the first system that has an available resource.

In the following example, assume that there are four shadow sets, and the SHADOW_MAX COPY parameter on this system is equal to 1. Note that the for shadow sets that are not assigned a specific level, a default priority level is assigned.

For the following example, assume that:

The user adds a device to DSA1. This is not a significant event, and this system does not interrupt the full copy operation of the DSA22 in favor of performing the DSA1 full copy operation.

To expand on this example, assume that a system fails (a significant event) before the copy operations have completed. All shadow sets are put into a merge required state. Specifically, DSA1, DSA20, and DSA22 are put into a full merge state, and DSA42 is put into a minimerge state.

After the significant event recovery delay expires, this system begins to evaluate all the shadow sets in a transient state. The operations take place in the following order:

Thus, in this example, the priority level is used to direct the priority of merge and copy operations on this system.

SHADOW_MAX_COPY is a dynamic system parameter that governs the use of system resources by shadowing. Shadowing can be directed to immediately respond to changes in this parameter setting using the following DCL command:

$ SET SHADOW/EVALUATE=RESOURCES

This command stops all the current merge and copy operations on the system on which it is issued. It then restarts the work using the new value of SHADOW_MAX_COPY.

This command is also useful in other circumstances. For example, if a shadow set has a priority level 0 or another low value, the SET SHADOW /PRIORITY=n command can be used to increase the value. Then, using the /EVALUATE=RESOURCES qualifier, the priority of shadow sets in a transient state is reevaluated.

The /PRIORITY and /EVALUATE=RESOURCES qualifiers can be used on the same command line.

When a significant event occurs, all of the SHADOW_MAX_COPY resources are applied. If the value of SHADOW_MAX_COPY is modified using the SYSGEN SET and WRITE ACTIVE commands, and then a SET SHADOW/EVALUATE=RESOURCES is issued, the new value of SHADOW_MAX_COPY has a direct and immediate affect.

To determine which system is controlling a transient operation, enter the following command:

$ SHOW SHADOW/ACTIVE DSAn:

To determine the priority values assigned to each shadow set, enter the following command:

$ SHOW SHADOW/BY_PRIORITY DSAn:

To remove an individual member from a shadow set, specify the name of the physical device with the DISMOUNT command. For example:

$DISMOUNT $5$DUA7:

When you dismount an individual shadow set member, all outstanding I/O operations are completed and the member is removed from the set.

The /FORCE_REMOVAL ddcu: qualifier is available. If connectivity to a device has been lost and the shadow set is in mount verification, /FORCE_REMOVAL ddcu: can be used to immediately expel a named shadow set member (ddcu: ) from the shadow set. If you omit this qualifier, the device is not dismounted until mount verification completes. Note that this qualifier cannot be used in conjunction with the /POLICY=[NO]MINICOPY [=OPTIONAL] qualifier.

The device specified must be a member of a shadow set that is mounted on the node where the command is issued.

The /FORCE_REMOVAL qualifier gives system managers greater control of shadow sets whose members are located at different sites in an OpenVMS Cluster configuration. SET SHADOW command qualifiers are also available for specifying management attributes for shadow set members located at the same or different sites, as described in Section 4.7 and in Section 4.8.

%DISM-F-SRCMEM, Only source member of shadow set cannot be dismounted

The way you dissolve a shadow set depends on whether it is mounted on a single system or on two or more systems in an OpenVMS Cluster system. In both cases, you use the DISMOUNT command. If the shadow set is mounted on a single system, you can dissolve the shadow set by specifying its virtual unit name with the DISMOUNT command. If the shadow set is mounted in a cluster, you must include the /CLUSTER qualifier to dissolve the DSA36 shadow set across the cluster. For example:

$ DISMOUNT /CLUSTER DSA36:

Dismounting the shadow set can be done only after all files are closed, thereby ensuring that the dismounted disks are fully consistent from a file system perspective. The dismount operation marks the shadow set members as being properly dismounted so that a rebuild is not required the next time the disks are mounted. However, if a merge operation was either pending or in progress, then the dismount operation marks the shadow set members as being improperly dismounted and requires a merge operation.

$DISMOUNT DSA9999:

%%%%%%%%%%% OPCOM 24-MAR-1990 20:29:57.52 %%%%%%%%%%%
$7$DUA6: (WRKDSK) has been removed from shadow set.
%%%%%%%%%%% OPCOM 24-MAR-1990 20:29:57.68 %%%%%%%%%%%
$7$DUA56: (PLADSK) has been removed from shadow set.
%%%%%%%%%%% OPCOM 24-MAR-1990 20:29:57.88 %%%%%%%%%%%
Message from user SYSTEM on SYSTMX

Site-specific shutdown command procedures can be created for each system in your cluster, as described in the OpenVMS System Manager’s Manual. The default SHUTDOWN.COM procedure that ships with the operating system performs a DISMOUNT/ABORT/OVERRIDE=CHECKS operation on all mounted volumes. If files are left open on any mounted shadow sets, a merge operation is required for these shadow sets when the system is rebooted.

To prevent such unnecessary merge operations, VSI recommends that you modify each site-specific SYSHUTDWN.COM command procedure to dismount the shadow sets without using the DISMOUNT/ABORT/OVERRIDE=CHECKS qualifiers. If open files are found, they should be closed.

As discussed in Section 4.10.2, the virtual unit can be dismounted on the system or across an OpenVMS Cluster system. To ensure that the virtual unit has been dismounted correctly, the following steps are recommended:

Use SHOW DEVICE in the following format to display information about shadow sets:

SHOW DEVICE [virtual-unit-name[:]]

The variable virtual-unit-name replaces device-name as the SHOW DEVICE command parameter for shadow sets. Use the virtual unit naming format DSAn:.

As with any SHOW DEVICE command, the colon is optional. Note also that you can specify a complete virtual unit name (or a portion of a virtual unit name) just as you can with device names. If you omit the virtual unit number, SHOW DEVICE lists all the shadow set virtual units that represent shadow set member disks of the type specified. If you truncate a device name (for example, if you specify D), SHOW DEVICE lists all the devices and all the virtual units that begin with the letters you entered (in this case, D).

When you specify the virtual unit number, SHOW DEVICE displays the names of the shadow set members it represents. If you use the /FULL qualifier, SHOW DEVICE displays full information about the shadow set and all the associated shadow set members.

Because individual shadow set members that are mounted for systemwide or clusterwide access are not allocated or mounted in the traditional sense, a SHOW DEVICE command with the /ALLOCATED or /MOUNTED qualifiers displays only virtual units.

Use the same format for the SHOW DEVICE command with shadow set members as you use with other physical devices. The command lists all shadow set members of the device name you specify.

Because shadow set members are not mounted in a traditional sense and they all have the same device characteristics, SHOW DEVICE displays most of the relevant data with the associated virtual unit. Listings of shadow set members include information about current membership status.

If a shadow set is undergoing a copy or a merge operation, the display resulting from the SHOW DEVICE command includes the percentage of the disk that has been copied or merged. The SHOW DEVICE information is available on all nodes that have the shadow set mounted.

The SHOW DEVICE display indicates the exact percentage of the disk that has been copied. The node that is managing the copy operation knows precisely how far the copy or merge operation has progressed, and periodically notifies the other nodes in the OpenVMS Cluster of the progress. Thus, the other nodes in the cluster know approximately the percentage copied. When you enter the SHOW DEVICE command from a node other than the one where the copy or merge operation is taking place, the number indicating the percentage copied in the SHOW DEVICE output lags (by a small percentage) the actual percentage copied.

Note that if a copy and a merge operation are occurring at the same time in the same shadow set, the number indicating the percentage merged remains static until the copy completes. Then the merge operation proceeds to completion.

The following examples of output from the SHOW DEVICE command illustrate the types of shadow set information you can obtain, such as shadow set membership and the status of each shadow set member during copy and merge operations. For examples of output for write bitmaps used with the minicopy operation, see Section 7.9.

Examples

$ SHOW DEVICE D
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA0:                Mounted           0  SHADOWDISK        8694   151   1
DSA9999:             Mounted           0  APPARITION      292971     1   1
$4$DUA0:   (SYSTMX)  Online            0
$4$DUA8:   (HSJ001)  ShadowSetMember   0  (member of DSA0:)
$4$DUA10:  (SYSTMX)  ShadowSetMember   0  (member of DSA9999:)
$4$DUA11:  (SYSTMX)  ShadowSetMember   0  (member of DSA9999:)
$4$DUA12:  (SYSTMX)  ShadowSetMember   0  (member of DSA9999:)
$4$DUA89:  (HSJ002)  ShadowSetMember   0  (member of DSA0:)

By truncating the device name, you cause the SHOW DEVICE command to list all the devices and all the virtual units on the local node that begin with the letters you entered (in this case, D). This example shows that two virtual units, DSA0 and DSA9999, are active. Both shadow sets are in a steady state. The device status “ShadowSetMember” indicates that the shadow set is in a steady state—the shadow set members are consistent with each other.

$ SHOW DEVICE DSA8
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA8:                Mounted           0  APPARITION      890937     1   1
$11$DUA8:   (SYSTMX)   ShadowSetMember   0  (member of DSA8:)
$11$DUA89:  (SYSTMY)   ShadowSetMember   0  (member of DSA8:)

This example shows the membership and status of the shadow set represented by the DSA8 virtual unit. The SHOW DEVICE display provides information not only about the virtual unit DSA8, but also about the physical devices $11$DUA8 and $11$DUA89 that are members of the shadow set. The device status “ShadowSetMember” indicates that the shadow set is in a steady state—the shadow set members are consistent with each other. The shadow set members are being served by OpenVMS Cluster nodes SYSTMX and SYSTMY.

$ SHOW DEVICE DSA
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA7:                Mounted           0  PHANTOM          27060    35   7
DSA8:                Mounted           0  APPARITION      890937     4   6

You might specify DSA on the SHOW DEVICE command to request information about all the shadow sets on the local node. Entering a generic virtual unit name, such as DSA, as a parameter produces a display of all virtual units representing shadow sets mounted on the local system. This example shows that two shadow sets are mounted on the local node, represented by the virtual units DSA7 and DSA8.

$ SHOW DEVICE $11$DUA8:
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA8:                Mounted           0  APPARITION      890937     1   1
$11$DUA8:   (HSJ001) ShadowSetMember   0  (member of DSA8:)
$11$DUA89:  (HSJ002) ShadowSetMember   0  (member of DSA8:)

Although the SHOW DEVICE command specifies the name of a single device, the resulting display includes information about the membership and status of the shadow set represented by the DSA8 virtual unit to which the $11$DUA8 device belongs. The device status “ShadowSetMember” indicates that the shadow set is in a steady state—the shadow set members are consistent with each other. The shadow set members are accessed through the node named HSJ001.

$ SHOW DEVICE $11$DUA8:
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA8:                Mounted           0  APPARITION      890937     1   1
$11$DUA8:  (HSJ001)  ShadowSetMember   0  (member of DSA8:)

$11$DUA89: (HSJ002)  ShadowCopying     0  (copy trgt DSA8:  48% copied)

The output from this SHOW DEVICE command shows a shadow set that is in a transient state. The device status “ShadowCopying” indicates that the physical device $11$DUA89 is the target of a copy operation, and 48% of the disk has been copied. The device $11$DUA8 is the source member for the copy operation.

$ SHOW DEVICE DSA8
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA8:                Mounted           0  APPARITION      890937     1  12

$11$DUA8:  (HSJ001)  ShadowCopying     0  (copy trgt DSA8: 5% copied)
$11$DUA89: (HSJ002)  ShadowMergeMbr    0  (merging DSA8:   0% merged)

This example shows how the SHOW DEVICE command displays a shadow set during a copy operation after a node in an OpenVMS Cluster system fails. In this example, the shadow set members are located on different nodes in the cluster, and one node on which the shadow set is mounted fails. At the time of the failure, the shadow set was in a transient state, with the $11$DUA8 device undergoing a copy operation. The SHOW DEVICE command shows the state of the shadow set during the copy operation, before the merge operation occurs.

At the same time the $11$DUA89 shadow set member is acting as the source member for the copy operation, $11$DUA89 also accepts and performs I/O requests from applications running on the OpenVMS Cluster system. Once the copy operation completes, a merge operation automatically starts. See Chapter 6 for more information about merge operations.

The next example shows how the SHOW DEVICE command display looks during the merge operation.

$ SHOW DEVICE DSA8
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA8:                Mounted           0  APPARITION      890937     1   1

$11$DUA8:   (HSJ001) ShadowMergeMbr    0  (merging DSA8: 78% merged)
$11$DUA89:  (HSJ002) ShadowMergeMbr    0  (merging DSA8: 78% merged)

The SHOW DEVICE command produces a display similar to this example when a shadow set is in a transient state because of a merge operation. The merge operation is 78% complete.

$ SHOW DEV D
Device               Device        Error    Volume         Free  Trans Mnt
 Name                Status        Count     Label        Blocks Count Cnt
DSA456:      (FUSS)  Mounted           0  AUDITINGDISK    123189   225  17
$11$DIA1:  (LISBEN)  Online            0
$11$DJA16: (GALEXI)  Online            0
$11$DJA128:(GALEXI)  Mounted wrtlck    0  CORPORATEVOL    164367     1  18
$11$DJA134:(GALEXI)  Mounted           0  WORKVOLUME      250344     1  16
$11$DUA1:    (FUSS)  Mounted           0  MAR24DISKVOL    676890     1  18
$11$DUA2:    (FUSS)  ShadowSetMember   0  (member of DSA456:)
$11$DUA7:   (BLISS)  Online            0  (remote shadow member)
$11$DUA11: (LISBEN)  Mounted           0  RMSFILES        621183     1  18
$11$DUA13:  (BLISS)  Mounted           0  RESIDENTVOL     525375     1  18

This example shows how the SHOW DEVICE command displays remote shadow set members. In this display, the device $11$DUA7, whose description is “remote shadow member,” is a member of a shadow set that is not mounted on this system.

$ SHOW DEVICE/FULL DSA80
Disk DSA80:, device type MSCP served SCSI disk, is online, mounted, file-
oriented device, shareable, available to cluster, error logging is enabled.

Error count                    0    Operations completed               138
Owner process                 ""    Owner UIC                      [SHADOW]
Owner process ID        00000000    Dev Prot    S:RWED,O:RWED,G:RWED,W:RWED
Reference count                1    Default buffer size                512
Total blocks              891072    Sectors per track                   51
Total cylinders             1248    Tracks per cylinder                 14
Volume label         "SHADTEST1"    Relative volume number               0
Cluster size                   3    Transaction count                    1
Free blocks               890937    Maximum files allowed           111384
Extend quantity                5    Mount count                          4
Mount status              System    Cache name         "_DSA2010:XQPCACHE"
Extent cache size             64    Maximum blocks in extent cache   89093
File ID cache size            64    Blocks currently in extent cache     0
Quota cache size               0    Maximum buffers in FCP cache       216
Volume status:  subject to mount verification, file high-water marking,write-
  through caching enabled.
Volume is also mounted on BLASTA, CNASTA, SHASTA.
Disk $255$DUA56:, device type MSCP served SCSI disk, is online, member of
shadow set DSA80:, error logging is enabled.
Error count                    0    Shadow member operation count      301
Host name               "SHASTA"    Host type, avail                   yes
Allocation class             255
  Volume status:  volume is a merge member of the shadow set.
Disk $255$DUA58:, device type MSCP served SCSI disk, is online, member of
shadow set DSA80:, error logging is enabled.
Error count                    0    Shadow member operation count      107
Host name               "SHASTA"    Host type, avail                   yes
Allocation class             255
Volume status:  volume is a merge member of the shadow set.

This example shows how the SHOW DEVICE/FULL command displays detailed information about the shadow set and its members. Notice that both members, $255$DUA56 and $255$DUA58, are merge members. Section 4.11.5 shows what this shadow set looks like when it is examined using the System Dump Analyzer.

The /SHADOW qualifier for the ANALYZE/DISK utility can be used to compare either a specified range of blocks in a shadow set or the entire contents of a shadow set. The ANALYZE/DISK/SHADOW command is useful if the INITIALIZE/SHADOW command was used without the /ERASE qualifier to initialize a shadow set. Another use of ANALYZE/DISK/SHADOW is to exercise the I/O subsystem.

In the unlikely event a discrepancy is found, the shadowset's clusterwide write lock is taken on the shadow set, and the blocks are reread. If a discrepancy is still present, the file name is displayed and the data block containing the discrepancy is dumped to the screen or to a file if /OUTPUT was specified. If no discrepancy is found on the second read, then the error is considered transient (a write was in flight to that disk block). Although the transient error is logged in the summary, verification that all members contained the same information is considered a success.

Differences outside the file system are expected if INITIALIZE/SHADOW was used without the /ERASE qualifier to initialize a shadow set. This is not disk corruption. The blocks that are reported as different have not been written to, but they may contain stale data. The blocks reported as inconsistent may even be allocated to a file, because there may be unwritten space between the file’s end-of-data location and the end of the allocated space.

To eliminate such inconsistencies, perform a full merge. To initiate a full merge, execute the DCL command SET SHADOW/DEMAND_MERGE DSAxxx. If the devices are served by controllers that support controller-based minimerge (for example, HSJ50s), this command should be issued while the shadow set is mounted on only one node within the cluster. Otherwise, a minimerge occurs, and the discrepancy may not be resolved. When you are adding members to a single member shadow set, a full copy operation also ensures that the disk is consistent both within and outside of the file system. If errors are reported on an ANALYZE/DISK/SHADOW command after a full merge has been executed, they should be investigated.

Differences are also expected in the following system files:

Table 4.4 describes the qualifiers for the ANALYZE/DISK/SHADOW command.

Example 4.9 shows the use of the ANALYZE/DISK/SHADOW command with the /BRIEF and /BLOCK qualifiers.

$ ANALYZE/DISK/SHADOW/BRIEF/BLOCK=COUNT=1000 DSA716:
Starting to check _DSA716: at 14-MAY-2003 13:42:52.43
Members of shadow set _DSA716: are _$252$MDA0: _$252$DUA716:
and the number of blocks to be compared is 1000.
Checking LBN #0 (approx 0%)
Checking LBN #127 (approx 12%)
Checking LBN #254 (approx 25%)
Checking LBN #381 (approx 38%)
Checking LBN #508 (approx 50%)
Checking LBN #635 (approx 63%)
Checking LBN #762 (approx 76%)
Checking LBN #889 (approx 88%)
Run statistics for _DSA716: are as follows:
         Finish Time = 14-MAY-2003 13:42:52.73
         ELAPSED TIME = 0 00:00:00.29
         CPU TIME = 0:00:00.02
         BUFFERED I/O COUNT = 10
         DIRECT I/O COUNT = 16
         Failed LBNs = 0
         Transient LBN compare errors = 0

The System Dump Analyzer (SDA) is a utility provided with the OpenVMS operating system. Although the main function of SDA is for crash dump analysis, it is also a useful tool for examining a running system, including the shadow sets. You can also use SDA to determine whether or not a third-party SCSI device supports the shadowing data repair (disk bad block errors) capability.An example is included in Section 4.11.5.1.

The SDA command SHOW DEVICE displays information from the system data structures that describe the devices in the system configuration. To examine a shadow set, first enter ANALYZE/SYSTEM at the DCL prompt to invoke the System Dump Analyzer. Then, at the SDA> prompt, enter the SHOW DEVICE command followed by the virtual unit name.

The following example shows how to obtain information about the shadow set represented by the virtual unit DSA80. Compare the SDA output in the following example with the DCL SHOW DEVICE output shown in the last example in Section 4.11.3.

$ ANALYZE/SYSTEM
VMS System analyzer
SDA> SHOW DEVICE DSA80
I/O data structures
-------------------
DSA80                                   HSJ00              UCB address:  810B7F50
Device status:   00021810 online,valid,unload,lcl_valid
Characteristics: 1C4D4008 dir,fod,shr,avl,mnt,elg,idv,odv,rnd
                 00082021 clu,mscp,loc,vrt
Owner UIC [004000,000015]   Operation count        138   ORB address    810B8080
      PID        00000000   Error count              0   DDB address    813F49F0
Alloc. lock ID   009C2595   Reference count          1   DDT address    810BEBB8
Alloc. class            0   Online count             1   VCB address    810BE3F0
Class/Type          01/15   BOFF                  0000   CRB address    8129EB10
Def. buf. size        512   Byte count            0200   PDT address    810121A0
DEVDEPEND        04E00E33   SVAPTE            81FDE55C   CDDB address   813F4360
DEVDEPND2        00000000   DEVSTS                0004   SHAD address   8111D460
FLCK index             34   RWAITCNT              0000   I/O wait queue    empty
DLCK address     00000000
Shadow Device status:   0004 nocnvrt
  ----- Shadow Descriptor Block (SHAD) 8111D460 -----
Virtual Unit status:              0041 normal,merging
Members                2    Act user IRPs          0    VU UCB          810B7F50
Devices                2    SCB LBN         0006CC63    Write log addr  00000000
Fcpy Targets           0    Generation Num  28D47C20    Master FL          empty
Mcpy Targets           2                    00935BC7    Restart FL         empty
Last Read Index        1    Virtual Unit Id 00000000
Master Index           0                    12610050
     ----- SHAD Device summary for Virtual Unit  DSA80  -----
Device $255$DUA56
 Index 0 Device Status    A6 merge,cip,src,valid
 UCB 810510D0     VCB 81400A00      Unit Id. 12A10038 000000FF
 Merge LBN 0004B94D
Device $255$DUA58
 Index 1 Device Status    A6 merge,cip,src,valid
 UCB 81051260     VCB 81439800      Unit Id. 12A1003A 000000FF
 Merge LBN 0004B94D
SDA> exit

The SDA utility's SHOW DEVICE command first displays device characteristics of the DSA80 virtual unit and the addresses of data structures. SDA then displays the DSA80 virtual unit status and the status of the individual shadow set members. Notice how the device status for each member reflects that the unit is in a merge state. For example, $255$DUA56 is shown with the following device status:

Device $255$DUA56
 Index 0 Device Status    A6 merge (1), cip (2), src (3), valid (4)
 UCB 810510D0     VCB 81400A00      Unit Id. 12A10038 000000FF
 Merge LBN 0004B94D

This information translates to the following:

Notice also how both devices $255$DUA56 and $255$DUA58 show that, at the time the SDA took this “snapshot” of the shadow set, the merge operation is merging at LBN 0004B94D.

The following example shows an SDA display of the same shadow set when $255$DUA56 is a merge member and $255$DUA58 is the recipient of a copy operation. A shadow set can be in this merge/copy state when a node that has the shadow set mounted crashes while a member in the shadow set is undergoing a copy operation. Volume shadowing automatically marks the member undergoing the copy operation so that it receives a merge operation after the copy operation completes. This ensures consistency across the shadow set.

The example first shows output for one shadow set member, using the DCL command SHOW DEVICE $255$DUA58; then the example shows the output for the entire shadow set, using the SDA command SHOW DEVICE DSA80. (SDA is invoked by the ANALYZE/SYSTEM command.)

$ SHOW DEVICE $255$DUA58
Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label        Blocks Count Cnt
DSA80:                  Mounted              0  SHADTEST1       890937     1   3

$255$DUA56:   (SHASTA)  ShadowMergeMbr       0  (merging DSA80:   0% merged)
$255$DUA58:   (SHASTA)  ShadowCopying        0  (copy trgt DSA80: 9% copied )

$ ANALYZE/SYSTEM
VMS System analyzer
SDA> SHOW DEVICE DSA80
I/O data structures
-------------------
DSA80                                   RA81              UCB address:  810B7F50
Device status:   00021810 online,valid,unload,lcl_valid
Characteristics: 1C4D4008 dir,fod,shr,avl,mnt,elg,idv,odv,rnd
                 00082021 clu,mscp,loc,vrt
Owner UIC [004000,000015]   Operation count        130   ORB address    810B8080
      PID        00000000   Error count              0   DDB address    813F49F0
Alloc. lock ID   009C2595   Reference count          1   DDT address    810BEBB8
Alloc. class            0   Online count             1   VCB address    810BE3F0
Class/Type          01/15   BOFF                  0000   CRB address    8129EB10
Def. buf. size        512   Byte count            0000   PDT address    810121A0
DEVDEPEND        04E00E33   SVAPTE            00000000   CDDB address   813F4360
DEVDEPND2        00000000   DEVSTS                0004   SHAD address   8111D460
FLCK index             34   RWAITCNT              0000   I/O wait queue    empty
DLCK address     00000000
Shadow Device status:   0004 nocnvrt
  ----- Shadow Descriptor Block (SHAD) 8111D460 -----
Virtual Unit status:              0061 normal,copying,merging
Members                1    Act user IRPs          0    VU UCB          810B7F50
Devices                2    SCB LBN         0006CC63    Master FL         empty
Fcpy Targets           1    Generation Num  7B7BE060    Restart FL         empty
Mcpy Targets           0                    00935BC4
Last Read Index        0    Virtual Unit Id 00000000
Master Index           0                    12610050
     ----- SHAD Device summary for Virtual Unit  DSA80  -----
Device $255$DUA56
 Index 0 Device Status    A2 merge,src,valid
 UCB 810510D0     VCB 81400A00      Unit Id. 12A10038 000000FF
 Merge LBN FFFFFFFF
Device $255$DUA58
 Index 1 Device Status    87 fcpy,merge,cip,valid
 UCB 81051260     VCB 81439800      Unit Id. 12A1003A 000000FF
 Copy LBN 00033671

In this example, in the SHAD Device summary for Virtual Unit DSA80 display, the device status (fcpy) for $255$DUA58 shows that it is the target of a full copy operation. The source of the operation is $255$DUA56; notice that the Merge LBN line for $255$DUA56 shows a series of Fs (FFFFFFFF). This notation indicates that a merge operation must be done after the copy operation completes. The Copy LBN line for the target disk $255$DUA58 shows that the copy operation is currently copying at LBN 00033671.

SDA> SHOW DEVICE DKA200:
I/O data structures
-------------------
COLOR$DKA200             Generic_DK        UCB address:  806EEAF0

Device status:   00021810 online,valid,unload,lcl_valid
Characteristics: 1C4D4008 dir,fod,shr,avl,mnt,elg,idv,odv,rnd
                 01010281 clu,srv,nnm,scsi

The F$GETDVI lexical function provides another method for obtaining information about devices mounted in shadow sets. Using F$GETDVI, you can obtain general device and volume information and specific information about the shadow set status of the device or volume. For example, you can determine the following types of information:

You can use the F$GETDVI lexical function interactively at the DCL command level or in a DCL command procedure. You can also use the $GETDVI system service with volume shadowing (see Section 5.6).

The format for the F$GETDVI lexical function is as follows:

F$GETDVI (device-name,item)

You supply two arguments to the F$GETDVI lexical function: a physical device name and the name of an item that specifies the type of information you want to obtain.

Table 4.5 lists the items specific to volume shadowing that you can supply as arguments to the F$GETDVI lexical function. It shows the type of information returned by each item and the data types of the return values. (The VSI OpenVMS DCL Dictionary: A-M lists all the item codes that you can supply as an argument to F$GETDVI.)

Example

To check a device for possible shadow set membership, you could include the following DCL command in a command procedure:

$  IF F$GETDVI("WRKD$:","SHDW_MEMBER") THEN GOTO SHADOW_MEMBER

If WRKD$ (a logical name for a disk) is a shadow set member, then F$GETDVI returns the string TRUE and directs the procedure to the volume labeled SHADOW_MEMBER.

See the VSI OpenVMS DCL Dictionary: A-M for additional information about the F$GETDVI lexical function.

If you want to remove a single member from a shadow set, you must make a call to $DISMOU. In the devnam argument, you should specify the name of the shadow set member you want to remove. The specified member is spun down unless you specify the DMT$M_NOUNLOAD option in the flags argument.

$DMTDEF
FLAGS:  .LONG DMT$M_NOUNLOAD
MEMBER001: .ASCID /$2$DUA9:/
 .
 .
 .

$DISMOU_S -
 devnam = MEMBER001, -
 flags = FLAGS
 .
 .
 .
.END

If you want to dismount a shadow set on a single node, you must make a call to $DISMOU. In the devnam argument, you should specify the name of the virtual unit that represents the shadow set you want to dismount. If you want to dismount the shadow set clusterwide, specify the DMT$M_CLUSTER option in the flags argument of the call.

When you dismount a shadow set on a single node in an OpenVMS Cluster system, and other nodes in the OpenVMS Cluster still have the shadow set mounted, none of the shadow set members contained in the shadow set are spun down, even if you have not specified the DMT$M_NOUNLOAD flag. After this call completes, the shadow set is unavailable on the node from which the call was made. The shadow set is still available to other nodes in the cluster that have the shadow set mounted.

If the node on which the shadow set is being dismounted is the only node that has the shadow set mounted, the shadow set dissolves. The shadow set member devices are spun down unless you specify the DMT$M_NOUNLOAD flag.

$DMTDEF
FLAGS:    .LONG 0
DSA23:  .ASCID /DSA23:/
 .
 .
 .

$DISMOU_S -
 devnam = DSA23, -
 flags = FLAGS
 .
 .
 .
.END

$DMTDEF
FLAGS:   .LONG  DMT$M_CLUSTER
DSA23:  .ASCID /DSA23:/
 .
 .
 .

$DISMOU_S -
 devnam = DSA23, -
 flags = FLAGS
 .
 .
 .
.END

You must specify the DMT$M_CLUSTER option with the flags argument if you want the shadow set dismounted from every node in the cluster. When each node in the cluster has dismounted the shadow set (the number of hosts having the shadow set mounted reaches zero), the volume shadowing software dissolves the shadow set.

To obtain the device names of all members of a shadow set, you must make a series of calls to $GETDVI. In your first call to $GETDVI, you can specify either the virtual unit that represents the shadow set or the device name of a member of the shadow set.

Unassisted copy operations are performed by an OpenVMS system. The actual transfer of data from the source member to the target is done through host node memory. Although unassisted copy operations are not CPU intensive, they are I/O intensive and consume a small amount of CPU bandwidth on the node that is managing the copy. An unassisted copy operation also consumes interconnect bandwidth.

On the system that manages the copy operation, user and copy I/Os compete evenly for the available I/O bandwidth. For other nodes in the cluster, user I/Os proceed normally and contend for resources in the controller with all the other nodes. Note that the copy operation may take longer as the user I/O load increases.

The volume shadowing software performs an unassisted copy operation when it is not possible to use the assisted copy feature (see Section 6.2.2). The most common cause of an unassisted copy operation is when the source and target disk or disks are not on line to the same controller subsystem. For unassisted copy operations, two disks can be active targets of an unassisted copy operation simultaneously, if the members are added to the shadow set on the same command line. Disks participating in an unassisted copy operation may be on line to any controller anywhere in a cluster.

During any copy operation, a logical barrier is created that moves across the disk, separating the copied and uncopied LBN areas. This barrier is known as a copy fence. The node that is managing the copy operation knows the precise location of the fence and periodically notifies the other nodes in the cluster of the fence location. Thus, if the node performing the copy operation shuts down, another node can continue the operation without restarting at the beginning. During a copy operation, the I/O requests are processed as follows:

The time and amount of I/O required to complete an unassisted copy operation depends heavily on the similarities of the data on the source and target disks. It can take at least two and a half times longer to copy a member containing dissimilar data than it does to complete a copy operation on a member containing similar data.

An assisted copy does not transfer data through the host node memory. The actual transfer of data is performed within the controller, by direct disk-to-disk data transfers, without having the data pass through host node memory. Thus, the assisted copy decreases the impact on the system, the I/O bandwidth consumption, and the time required for copy operations.

Shadow set members must be accessed from the same controller in order to take advantage of the assisted copy. The shadowing software controls the copy operation by using special MSCP copy commands, called disk copy data (DCD) commands, to instruct the controller to copy specific ranges of LBNs. For an assisted copy, only one disk can be an active target for a copy at a time.

For OpenVMS Cluster configurations, the node that is managing the copy operation issues an MSCP DCD command to the controller for each LBN range. The controller then performs the disk-to-disk copy, thus avoiding consumption of interconnect bandwidth.

By default, the Volume Shadowing for OpenVMS software (beginning with OpenVMS Version 5.5–2) and the controller automatically enable the copy assist if the source and target disks are accessed through the same HSC or HSJ controller.

See Section 6.4 for information about disabling and reenabling the assisted copy capability.

For systems running software earlier than OpenVMS Version 5.5–2, the merge operation is performed by the system and is known as an unassisted merge operation.

To ensure minimal impact on user I/O requests, volume shadowing implements a mechanism that causes the merge operation to give priority to user and application I/O requests.

The shadow server process performs merge operations as a background process, ensuring that when failures occur, they minimally impact user I/O. A side effect of this is that unassisted merge operations can often take an extended period of time to complete, depending on user I/O rates. Also, if another node fails before a merge completes, the current merge is abandoned and a new one is initiated from the beginning.

Note that data availability and integrity are fully preserved during merge operations regardless of their duration. All shadow set members contain equally valid data.

Starting with OpenVMS Version 5.5–2, the merge operation includes enhancements for shadow set members that are configured on controllers that implement assisted merge capabilities. The assisted merge operation is also referred to as a minimerge. The minimerge feature significantly reduces the amount of time needed to perform merge operations. Usually, the minimerge completes in a few minutes. For more information, see Chapter 8.

By using information about write operations that were logged in controller memory, the minimerge is able to merge only those areas of the shadow set where write activity was known to have been in progress. This avoids the need for the total read and compare scans required by unassisted merge operations, thus reducing consumption of system I/O resources.

The minimerge operation is enabled on nodes running OpenVMS Version 5.5–2 or later. Volume shadowing automatically enables the minimerge if the controllers involved in accessing the physical members of the shadow set support it. See the Volume Shadowing for OpenVMS Software Product Description (SPD 27.29. xx) for a list of supported controllers. Note that minimerge operations are possible even when shadow set members are connected to different controllers. This is because write log entries are maintained on a per controller basis for each shadow set member.

To create a bitmap, you must specify the /POLICY=MINICOPY[=OPTIONAL] qualifier with the DISMOUNT command. If you specify /POLICY=MINICOPY=OPTIONAL, a bitmap is created if there is sufficient memory. The disk is dismounted, regardless of whether a bitmap is created.

$ DISMOUNT $4$DUA1 /POLICY=MINICOPY=OPTIONAL

This command removes $4$DUA1 from the shadow set and starts logging writes to a bitmap, if possible.

If you specify /POLICY=MINICOPY only (that is, if you omit =OPTIONAL) and there is not enough memory on the node to create a bitmap, the dismount fails.

The bitmap created with this command is used for a minicopy operation when you later mount one of the former members of the shadow set into the set.

If you specify the /POLICY=MINICOPY=OPTIONAL qualifier and the shadow set is already mounted on another node in the cluster, the MOUNT command succeeds but a bitmap is not created.

You can find out whether a write bitmap exists for a shadow set by using the DCL command SHOW DEVICE/FULL device-name. If a shadow set supports write bitmaps, device supports bitmaps is displayed along with either bitmaps active or no bitmaps active. If the device does not support write bitmaps, no message pertaining to write bitmaps is displayed.

$ SHOW DEVICE/FULL DSA0

Disk DSA0:, device type RAM Disk, is online, mounted, file-oriented device,
    shareable, available to cluster, error logging is enabled, device supports     bitmaps (no bitmaps active).

 Error count               0    Operations completed               47
 Owner process            ""    Owner UIC                    [SYSTEM]
 Owner process ID   00000000    Dev Prot          S:RWPL,O:RWPL,G:R,W
 Reference count           2    Default buffer size               512
 Total blocks           1000    Sectors per track                  64
 Total cylinders           1    Tracks per cylinder                32
 Volume label         "TST0"    Relative volume number              0
 Cluster size              1    Transaction count                   1
 Free blocks             969    Maximum files allowed             250
 Extend quantity           5    Mount count                         1
 Mount status         System    Cache name    "_$252$DUA721:XQPCACHE"
 Extent cache size        64    Maximum blocks in extent cache     96
 File ID cache size       64    Blocks currently in extent cache    0
 Quota cache size          0    Maximum buffers in FCP cache      404
 Volume owner UIC   [SYSTEM]    Vol Prot  S:RWCD,O:RWCD,G:RWCD,W:RWCD

  Volume Status:  ODS-2, subject to mount verification, file high-water marking, write-back caching enabled.

Disk $252$MDA0:, device type RAM Disk, is online, member of shadow set DSA0:.

 Error count                    0    Shadow member operation count       128
 Allocation class             252


Disk $252$MDA1:, device type RAM Disk, is online, member of shadow set DSA0:.

 Error count                    0    Shadow member operation count       157
 Allocation class             25

You can find out the ID of each bitmap on a node with the DCL command SHOW DEVICE/BITMAP device-name. The /BITMAP qualifier cannot be combined with other SHOW DEVICE qualifiers except /FULL. The SHOW DEVICE/BITMAP display can be brief or full; brief is the default.

If no bitmap is active, no bitmap ID is displayed. The phrase no bitmaps active is displayed.

$ SHOW DEVICE/BITMAP DSA1
Device     BitMap      Size     Percent of
 Name        ID       (Bytes)    Full Copy
DSA1:      00010001     652        11%

The following example shows a SHOW DEVICE/BITMAP/FULL display:

$ SHOW DEVICE DSA12/BITMAP/FULL
Device  Bitmap  Size   Percent of  Active Creation  Master  Cluster Local Delete  Bitmap
Name      ID   (bytes) Full Copy     Date/Time       Node    Size    Set   Pending Name
DSA12: 00010001  652      11%      Yes  5-MAY-2000  13:30... 300F2   127     2%     No   SHAD$TEST

$ SHOW CLUSTER/CONTINUOUS

Command > ADD BITMAPS
Command > ADD CSID

View of Cluster from system ID 57348  node: WPCM1   14-FEB-2000 13:38:53

      SYSTEMS              MEMBERS
  NODE   SOFTWARE    CSID   STATUS    BITMAPS

 CSGF1   VMS X6TF    300F2   MEMBER    MINICOPY

 HSD30Y  HSD YA01    300E6

 HS1CP2  HSD V31D    300F4

 CSGF2   VMS X6TF    300D0   MEMBER    MINICOPY

In this example, MINICOPY means that nodes CSGF1 and CSGF2 are capable of supporting minicopy operations. If a cluster node does not support minicopy, the term UNSUPPORTED replaces MINICOPY in the display, and the minicopy function is disabled in the cluster.

After a minicopy operation is completed, the corresponding bitmap is automatically deleted.

$ DELETE/BITMAP/LOG 00010001
%DELETE-I-DELETED, 00010001 deleted

Removal of a shadow set member results in what is called a crash-consistent copy. That is, the copy of the data on the removed member is of the same level of consistency as what would result if the system had failed at that instant. The ability to recover from a crash-consistent copy is ensured by a combination of application design, system and database design, and operational procedures. The procedures to ensure recoverability depend on application and system design and will be different for each site.

The conditions that might exist at the time of a system failure range from no data having been written, to writes that occurred but were not yet written to disk, to all data having been written. The following sections describe components and actions of the operating system that may be involved if a failure occurs and there are outstanding writes, that is, writes that occurred but were not written to disk. You must consider these issues when establishing procedures to ensure data consistency in your environment.

To achieve data consistency, application activity should be suspended and no operations should be in progress. Operations in progress can result in inconsistencies in the backed-up application data. While many interactive applications tend to become quiet if there is no user activity, the reliable suspension of application activity requires cooperation in the application itself. Journalling and transaction techniques can be used to address in-progress inconsistencies but must be used with extreme care. In addition to specific applications, miscellaneous interactive use of the system that might affect the data to be backed up must also be suspended.

Applications that use RMS file access must be aware of the following issues.

OpenVMS allows access to files as backing store for virtual memory through the process and global section services. In this mode of access, the virtual address space of the process acts as a cache on the file data. OpenVMS provides the $UPDSEC service to force updates to the backing file.

ALTER TABLESPACE tablespace-name BEGIN BACKUP;

ALTER TABLESPACE tablespace-name END BACKUP;

It is critical to back up the database logs and control files as well as the database data files.

The base OpenVMS file system caches free space. However, all file metadata operations (such as create and delete) are made with a “careful write-through” strategy so that the results are stable on disk before completion is reported to the application. Some free space may be lost, which can be recovered with an ordinary disk rebuild. If file operations are in progress at the instant the shadow member is dismounted, minor inconsistencies may result that can be repaired with ANALYZE/DISK. The careful write ordering ensures that any inconsistencies do not jeopardize file Integrity server before the disk is repaired.

OpenVMS maintains a virtual I/O cache (VIOC) to cache file data. However, this cache is write through. OpenVMS Version 7.3 introduces extended file cache (XFC), which is also write through.

File writes using the $QIO service are completed to disk before completion is reported to the caller.

Multiple shadow sets present the biggest challenge to splitting shadow sets for backup. While the removal of a single shadow set member is instantaneous, there is no way to remove members of multiple shadow sets simultaneously. If the data that must be backed up consistently spans multiple shadow sets, application activity must be suspended while all shadow set members are being dismounted. Otherwise, the data is not crash consistent across the multiple volumes. Command procedures or other automated techniques are recommended to speed the dismount of related shadow sets. If multiple shadow sets contain portions of an Oracle database, putting the database into backup mode ensures recoverability of the database.

The OpenVMS software RAID driver presents a special case for multiple shadow sets. A software RAID set may be constructed of multiple shadow sets, each consisting of multiple members. With the management functions of the software RAID driver, it is possible to dismount one member of each of the constituent shadow sets in an atomic operation. Management of shadow sets used under the RAID software must always be done using the RAID management commands to ensure consistency.

All management operations used to attain data consistency must be performed for all members of an OpenVMS Cluster system on which the affected applications are running.

Testing alone cannot guarantee the correctness of a backup procedure. However, testing is a critical component of designing any backup and recovery process.

Too often, organizations concentrate on the backup process with little thought to how their data is restored. Remember that the ultimate goal of any backup strategy is to recover data in the event of a disaster. Restore and recovery procedures must be designed and tested as carefully as the backup procedures.

The discussion in this section is based on features and behavior of OpenVMS Version 7.3 (and higher) and applies to prior versions as well. Future versions of OpenVMS may have additional features or different behavior that affect the procedures necessary for data consistency. Sites that upgrade to future versions of OpenVMS must reevaluate their procedures and be prepared to make changes or to employ nonstandard settings in OpenVMS to ensure that their backups remain consistent.

When a system with a mounted shadow set fails, if a write request is made to a shadow set and the system fails before a completion status is returned to the application, the data might be inconsistent on the shadow set members:

The exact timing of the failure during the original write request determines the outcome. Volume Shadowing for OpenVMS ensures that corresponding LBNs on each shadow set member contain the same data (old or new), when the application issues a read to the shadow set.

A shadow set that enters mount verification and either times out or aborts mount verification enters a merge state, if the following conditions are true:

The system on which the mount verification timed out (or aborted mount verification) notifies the other systems on which the shadow set is mounted that a merge operation is needed, and then it disables the shadow set. (It does not dismount it.)

For example, if a shadow set is mounted on eight systems and mount verification times out on two of them, those two systems check their internal queues for write I/O. If any write I/O is found, the shadow set is merged.

The SET SHADOW/DEMAND_MERGE command initiates a merge of a specified shadow set or of all shadow sets. This qualifier is useful if the shadow set is created with the INITIALIZE/SHADOW command without using the /ERASE qualifier.

For more information about using the SET SHADOW/DEMAND_MERGE command, see the VSI OpenVMS DCL Dictionary: N-Z.

In a full merge operation, the members of a shadow set are compared with each other to ensure that they contain the same data. This is done by performing a block-by-block comparison of the entire volume. This can be a very lengthy procedure.

A minimerge operation is significantly faster. By using information about write operations that are logged in volatile controller storage or in a write bitmap on an OpenVMS system, volume shadowing merges only those areas of the shadow set where the write activity occurred. This avoids the need for the entire volume scan that is required by full merge operations, thus reducing consumption of system I/O resources.

Minimerges existed as controller-based prior to the introduction of HBMM, and available only on the HSJ, HSC, and HSD controllers.

Volume Shadowing for OpenVMS Version 8.4 is enhanced to increase the performance of minicopy and minimerge by “looking ahead” of the next bit that is set in the write bitmap. The actual number of QIOs between the SHADOW_SERVER and SYS$SHDRIVER is reduced by this method allowing the minimerge and minicopy to complete fast.

For a given bitmap, there is one master version on a system in the cluster and a local version on every other system on which the shadow set is mounted. A minimerge operation can occur only on a system with a master bitmap. For OpenVMS Version 8.3 and earlier, a shadow set can have up to six HBMM master bitmaps. Starting from OpenVMS Version 8.4, a shadow set can have up to 12 HBMM master bitmaps. Multiple master bitmaps for the same shadow set are equivalent but they do have different bitmap IDs.

The following example shows two master bitmaps for DSA12, one on RAIN and one on SNOW, each with a unique bitmap ID:

$ SHOW DEVICE/BITMAP DSA12
Device         BitMap     Size     Percent     Type of   Master  Active
 Name            ID      (Bytes)  Populated    Bitmap     Node
DSA12:        00040013      4132      0.01%   Minimerge  RAIN    Yes
              00010014      4132      0.01%   Minimerge  SNOW    Yes

If only one master bitmap exists for the shadow set and the system with the master bitmap fails or is shut down, the remaining local bitmap versions are automatically deleted. Local bitmaps cannot be used for recovery.

If multiple master bitmaps are created for the shadow set and at least one remains, that master bitmap can be used for recovery. VSI recommends the use of multiple master bitmaps, especially for multiple-site cluster systems. Multiple master bitmaps increase the likelihood of an HBMM operation rather than a full merge in the event of a system failure.

Bitmaps require additional memory. The calculation is based on the shadow set volume size. For every gigabyte of storage of a shadow set mounted on a system, 2 KB of bitmap memory is required on that system for each bitmap. For example, a shadow set with a volume size of 200 GB of storage and 2 bitmaps uses 800 KB of memory on every system on which it is mounted.

An HBMM policy specifies the following attributes for one or more shadow sets:

You can assign a name to a policy. However, the reserved names DEFAULT and NODEFAULT have specific properties that are described in Section 8.4. You can also create a policy without a name and assign it to a specific shadow set. An advantage of a named policy is that it can be reused by specifying only its name.

Multiple policies can be created to customize the minimerge operations in a cluster.

You can use the SET SHADOW/POLICY command with HBMM-specific qualifiers to define, assign, de-assign, and delete policies and to enable and disable HBMM on a shadow set. SET SHADOW/POLICY is the only user interface command for specifying HBMM policies. You cannot use the MOUNT command to define a policy.

You can define a policy before the shadow set is mounted. Policies can be assigned to shadow sets in other ways as well, as described in Section 8.4.

Three system parameters, namely, SHADOW_REC_DLY, SHADOW_PSM_DLY, and SHADOW_HBMM_RTC support HBMM. For more information about these system parameters, see the Section 3.3.

There are several factors you must consider when selecting the number of master bitmaps to specify in a policy and the systems that host the master bitmaps. The first issue is how many master bitmaps must be used in the configuration. For OpenVMS Version 8.3 and earlier, six is the maximum number of HBMM master bitmaps per shadow set. Starting from OpenVMS Version 8.4, 12 is the maximum per shadow set. The use of each additional master bitmap has a slight impact on write performance and also consumes memory on each system (as described in Section 8.2.1).

Using only one master bitmap creates a single point of failure; if the system hosting the master bitmap fails, then this shadow set undergoes a full merge. Therefore, the memory consumption must be weighed against the adverse effects of a full merge. Multiple master bitmaps provide the greatest defense against performing full merges.

Another issue when selecting a system to host the master bitmap is the I/O bandwidth of the various systems. Keep in mind that minimerges are always performed on a system that has a master bitmap. Therefore, low-bandwidth systems, such as satellite cluster members, are not good candidates.

The disaster tolerance of the configuration is also important in the decision process. Specifying systems to host master bitmaps at multiple sites help ensure that a minimerge is performed if connectivity to an entire site is lost. A two-site configuration must ensure that half the master bitmap systems are at each site, and a three-site configuration must ensure that one third of the master bitmaps are at each of the three sites.

When selecting a threshold reset value, you must balance the effects of bitmap resets on I/O performance with the time it takes to perform HBMM minimerges. The goal is to set the reset value as low as possible (thus decreasing merge times) while not affecting application I/O performance. Too low a value degrades I/O performance. Too high a value causes merges to take extra time.

HBMM bitmaps keep track of writes to a shadow set. The more bits that are set in the bitmap, the greater the amount of merging that is required in the event of a minimerge. HBMM clears the bitmap (after ensuring that all outstanding writes have completed so that the members are consistent) when certain conditions are met (see the description of SHADOW_HBMM_RTC in Section 3.3). A freshly cleared bitmap, with few bits set, performs a minimerge faster.

The bitmap reset, however, can affect I/O performance. Before a bitmap reset can occur, all write I/O to the shadow set must be paused and any write I/O that is in progress must be completed. Then the bitmap is cleared. This is performed on all systems on a per shadow set basis. Therefore, avoid a reset threshold setting that causes frequent resets.

You can view the number of resets performed by using the SHOW SHADOW command. The HBMM Reset Count appears in the last section of the display, as shown in the following example:

$ SHOW SHADOW DSA1031
_DSA1031: Volume Label: HBMM1031
Virtual Unit State: Steady State
Enhanced Shadowing Features in use:
Host-Based Minimerge (HBMM)
VU Timeout Value 3600 VU Site Value 0
Copy/Merge Priority 5000 Mini Merge Enabled
Served Path Delay 30
HBMM Policy
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the systems: LEMON, ORANGE
Any 1 of the systems: MELON, PEACH
HBMM bitmaps are active on LEMON, MELON, ORANGE
HBMM Reset Count 2 Last Reset 13-JUL-2004 10:13:53.90
Modified blocks since last bitmap reset: 11181
.
.
.
$

Writes that need to set bits in the bitmap are slightly slower than writes to areas that are already marked as having been written. Therefore, if many of the writes to a particular shadow set are concentrated in certain “hot” files, the reset threshold must be made large enough so that the same bits are not constantly set and then cleared.

On the other hand, if the reset threshold is too large, then the advantages of HBMM are reduced. For example, if 50% of the bitmap is populated (that is, 50% of the shadow set has been written to since the last reset), the HBMM merge takes approximately 50% of the time of a full merge.

You can change the RESET_THRESHOLD value while a policy is in effect by specifying the same policy with a different RESET_THRESHOLD value. (The syntax for specifying a policy, including the RESET_THRESHOLD keyword, is described in Section 8.3.)

The following example shows how to:

$!
$! To display status information about DSA3233
$!
$ Show Shadow DSA3233
_DSA30:   Volume Label: OSCAR
  Virtual Unit State:   Steady State
  Enhanced Shadowing Features in use:
        Host-Based Minimerge (HBMM)
        Extended Memberships
  VU Timeout Value      3600    VU Site Value          0
  Copy/Merge Priority   3233    Mini Merge       Enabled
  Recovery Delay Per Served Member                    30
  Merge Delay Factor     200    Delay Threshold      200
  Device $1$DGA32
    Read Cost              2    Site 0
    Member Timeout       120
  Device $1$DGA33               Master Member
    Read Cost              2    Site 0
    Member Timeout       120
$!
$! To create a policy and assign it to DSA3233
$!
$ SET SHADOW/POLICY=HBMM=(master_list=(ATHRUZ,ATWOZ,A2ZIPF),count=2,-
reset_threshold=420000) DSA3233:
$!
$! To confirm that the policy was assigned to DSA3233
$!
$ Show Shadow DSA3233
_DSA3233:   Volume Label: DSA3233
  Virtual Unit State:     Steady State
  Enhanced Shadowing Features in use:
        Host-Based Minimerge (HBMM)
  VU Timeout Value     3600   VU Site Value    0
  Copy/Merge Priority  3233   Mini Merge       Enabled
  Recovery Delay Per Served Member             30
  Merge Delay Factor   200    Delay Threshold  200
  HBMM Policy
    HBMM Reset Threshold: 420000
    HBMM Master lists:
      Up to any 2 of the nodes: ATHRUZ,ATWOZ,A2ZIPF
    HBMM bitmaps are active on ATHRUZ,ATWOZ
    Modified blocks since bitmap creation: 0
  Device $1$DGA32
    Read Cost          2   Site 0
    Member Timeout   120
  Device $1$DGA33          Master Member
    Read Cost          2   Site 0
    Member Timeout   120
$!
$! To change the Reset Threshold value
$!
$ SET SHAD/POLICY=HBMM=(master_list=(ATHRUZ,ATWOZ,A2ZIPF),count=2, -
reset_threshold=840000) DSA3233:
$!
$! To confirm the change to the Reset Threshold value
$!
$ Show Shadow DSA3233
_DSA3233:   Volume Label: DSA3233
  Virtual Unit State:     Steady State
  Enhanced Shadowing Features in use:
        Host-Based Minimerge (HBMM)
  VU Timeout Value     3600   VU Site Value        0
  Copy/Merge Priority  3233   Mini Merge     Enabled
  Recovery Delay Per Served Member                30
  Merge Delay Factor    200   Delay Threshold    200
  HBMM Policy
    HBMM Reset Threshold: 840000
    HBMM Master lists:
      Up to any 2 of the nodes: ATHRUZ,ATWOZ,A2ZIPF
    HBMM bitmaps are active on ATHRUZ,ATWOZ
    Modified blocks since bitmap creation: 0
  Device $1$DGA32
    Read Cost           2    Site 0
    Member Timeout    120
  Device $1$DGA33            Master Memeber
    Read Cost           2    Site 0
    Member Timeout    120

HBMM policies are defined to implement decisions regarding master bitmaps. Some sites might find that a single policy can effectively implement the decisions. Other sites might require greater granularity and therefore implement multiple policies.

Multiple policies are required when the cluster includes enough high-bandwidth systems that you want to ensure that the merge load is spread out. Note that minimerges occur only on systems that host a master bitmap. Therefore, if 12 systems with high bandwidth are set up to perform minimerge or merge operations (the system parameter SHADOW_MAX_COPY is greater than zero on all systems), you must ensure that the master bitmaps are spread out among these high-bandwidth systems.

Multiple HBMM policies are also useful when shadow sets need different bitmap reset thresholds. The list of master bitmap systems can be the same for each policy, but the threshold can differ.

The SET SHADOW/POLICY=HBMM command is used to define HBMM policies. You can define multiple policies for your environment. The following examples show how to define two policies, a DEFAULT policy and POLICY_1, a named policy.

To define the policy named DEFAULT:

$ SET SHADOW/POLICY=HBMM=(MASTER_LIST=*)/NAME=DEFAULT

In this example, a DEFAULT policy is created for the cluster. The use of the asterisk wildcard (*) means that any system can host a master bitmap. The omission of the keyword COUNT=n means that up to six systems (the default value and the current maximum supported) can host a master bitmap. The DEFAULT policy is inherited at mount time by shadow sets that have not been assigned a named policy.

The following example defines a named policy (POLICY_1), specifies the systems that are eligible to host a master bitmap, limits to two the number of systems to host a master bitmap, and specifies a higher threshold (default is 1,000,000 blocks) to be reached before clearing the bitmap.

$ SET SHADOW /POLICY=HBMM=( -
_$ (MASTER_LIST=(NODE1,NODE2,NODE3), COUNT=2), -
_$ RESET_THRESHOLD=1250000) -
_$ /NAME=POLICY_1

In OpenVMS Version 8.4, if you are using the DISMOUNT keyword, you can have up to 12 HBMM master bitmaps. For DISMOUNT keyword examples, see the Section 8.10.2.

For the complete DCL syntax of the SET SHADOW/POLICY=HBMM command, see the VSI OpenVMS DCL Dictionary: N-Z.

You can assign a policy, named or unnamed, to a shadow set. To assign an existing named policy, use the following command:

$ SET SHADOW DSAn:/POLICY=HBMM=policy-name

To assign an unnamed policy to a shadow set, use the same command, but in place of the policy name, specify the attributes of the policy you want to use. For example:

$ SET SHADOW DSA1:/POLICY=HBMM=(MASTER_LIST=(NODE1, NODE2, NODE3), COUNT=2)

In this example, the default bitmap reset value of 1,000,000 blocks takes effect because the RESET_THRESHOLD keyword is omitted.

HBMM is automatically activated on a shadow set under the following conditions:

You can also activate HBMM using the SET SHADOW/ENABLE=HBMM command, provided a policy exists and the shadow set is mounted on a system defined in the master list of the shadow set policy, and the count has not been exceeded.

To disable HBMM on a shadow set, use the following command:

$ SET SHADOW DSAn:/DISABLE=HBMM

Reasons for disabling HBMM on a shadow set include:

HBMM remains disabled until you either re-enable it or define a new policy for the shadow set.

Before removing a policy assignment from a shadow set, HBMM must be disabled, if active. You can remove a policy assignment from a shadow set by entering the following command:

$ SET SHADOW DSAn:/POLICY=HBMM/DELETE

This command removes any policy set for this shadow set, making the shadow set eligible for the DEFAULT policy. If a DEFAULT policy exists, it is assigned the next time the shadow set is eligible for a policy, for example, at the end of a merge or when you issue the SET SHADOW/ENABLE=HBMM command.

To change a policy assigned to a shadow set, you must first disable HBMM, as described in Section 8.6.4, and then assign another policy to the shadow set. To apply a different policy, specify the policy name or specify the policy attributes (thereby creating an “unnamed” policy), as described in Section 8.6.2. Specifying a new policy (or policy attributes) for a shadow set replaces the previous policy. The use of the command shown in Section 8.6.5 is not required when you are changing the policy assignment.

You can delete a named policy with the /DELETE qualifier, as shown in the following example:

$ SET SHADOW /POLICY=HBMM/NAME=policy-name/DELETE

This command deletes the specified policy, which takes effect across the cluster. It does not delete the policy from any shadow set to which it is already assigned.

The DEFAULT policy can be changed at any time. However, if a previous definition of the DEFAULT policy is assigned to a shadow set, a subsequent change to the definition of the DEFAULT policy is not retroactively applied to that shadow set. In this regard, the DEFAULT policy behaves just like any other named policy.

This section shows how to apply a changed DEFAULT policy.

Initially, the following DEFAULT policy is assigned to DSA20 when it is mounted, as shown by the following example:

$ SET SHADOW/POLICY=HBMM=(MASTER=(NODE1,NODE2,NODE3),COUNT=2)/NAME=DEFAULT
$ MOUNT/SYSTEM DSA20:/SHADOW=($1$DGA20,$1$DGA21) VOL_20

Subsequently, the DEFAULT policy is redefined by the following command. This redefined policy allows any node in the cluster to be eligible for an HBMM master bitmap:

$ SET SHADOW/POLICY=HBMM=(MASTER=*,COUNT=2)/NAME=DEFAULT

You can apply the redefined DEFAULT policy to DSA20 using the following commands:

$ SET SHADOW DSA20:/DISABLE=HBMM
$ SET SHADOW DSA20:/POLICY=HBMM/DELETE
$ SET SHADOW DSA20:/ENABLE=HBMM

An alternative way to apply the updated DEFAULT policy to DSA20 is to take advantage of the fact that the DEFAULT policy is a named policy. This method requires only two commands, as shown:

$ SET SHADOW DSA20:/DISABLE=HBMM
$ SET SHADOW DSA20:/POLICY=HBMM=DEFAULT

You can display policies using the SHOW SHADOW command. You can display:

Displaying the Policy of a Specific Shadow Set

To display the policy assigned to a specific shadow set, issue the following command:

$ SHOW SHADOW DSAn:/POLICY=HBMM

An example of the resulting output:

$ SHOW SHADOW DSA999:/POLICY=HBMM
HBMM Policy for device _DSA999:
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5
Up to any 2 of the nodes: NODE6,NODE7,NODE8

Displaying the Definition of a Named Policy

To display the definition of a named policy, issue the following command:

$ SHOW SHADOW/POLICY=HBMM/NAME=policy-name

The following display shows the definition of the PEAKS_ISLAND policy:

$ SHOW SHADOW/POLICY=HBMM/NAME=PEAKS_ISLAND
HBMM Policy PEAKS_ISLAND
HBMM Reset Threshold: 750000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5
Up to any 2 of the nodes: NODE6,NODE7,NODE8

Displaying All Shadow Sets with Policy Assignments

To display all shadow sets in a cluster with policy assignments, along with the definition of each policy, use the following command:

$ SHOW SHADOW/POLICY=HBMM

The following display results from this command:

$ SHOW SHADOW/POLICY=HBMM
HBMM Policy for device _DSA12:
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2
HBMM bitmaps are active on NODE1,NODE2
Modified blocks since bitmap creation: 254

HBMM Policy for device _DSA30:
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: FLURRY,FREEZE,HOTTUB

HBMM Policy for device _DSA99:
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5
Up to any 2 of the nodes: NODE6,NODE7,NODE8

HBMM Policy for device _DSA999:
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5
Up to any 2 of the nodes: NODE6,NODE7,NODE8

Displaying All Named Policies on a Cluster

To display the named policies that exist on a cluster, along with their definitions, issue the following command:

$ SHOW SHADOW/POLICY=HBMM/NAME

The named policies are displayed in the order in which they were created. The following display results from this command:

$ SHOW SHADOW/POLICY=HBMM/NAME
HBMM Policy DEFAULT
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 6 nodes in the cluster

HBMM Policy PEAKS_ISLAND
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5
Up to any 2 of the nodes: NODE6,NODE7,NODE8

HBMM Policy POLICY_1
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: NODE1,NODE2,NODE3
Any 1 of the nodes: NODE4,NODE5

HBMM Policy ICE_HOTELS
HBMM Reset Threshold: 1000000
HBMM Master lists:
Up to any 2 of the nodes: QUEBEC,ICELND,SWEDEN
Any 1 of the nodes: ALASKA,GRNLND

You can check the merge status of each shadow set member by issuing the SHOW SHADOW/MERGE DSAn command. The /MERGE qualifier returns one of the following messages:

An example of the display produced by the SHOW SHADOW/MERGE DSAn command:

$ SHOW SHADOW/MERGE
Device    Volume     Device
 Name      Label     Status
_DSA1010  MINIMERGE  Merging (10%)

If a copy operation (instead of the merge operation) is currently active, the display shows the percentage of the merge that has completed and the percentage of the copy that has completed with the designation “Copy Active,” as follows:

$ SHOW SHADOW/MERGE
Device    Volume   Device
 Name      Label   Status
_DSA1010  FOOBAR   Merging (23%), Copy Active (77%) on CSGF1

You can prevent merge operations on a system in two ways:

Only systems that have an HBMM master bitmap for a particular shadow set are able to perform HBMM recovery on that shadow set. If a merge recovery is required on a shadow set and no systems in the cluster have an HBMM master bitmap for that shadow set, a full merge is performed.

Therefore, to minimize the need to perform a full merge, you must use policies that maintain at least one HBMM master bitmap at each site in a multiple-site OpenVMS Cluster system. The ability to specify multiple master lists in an HBMM policy is designed for this purpose. You must specify a separate MASTER_LIST for each site.

For example, consider a three-site OpenVMS Cluster system with 12 cluster members:

The following definition of a DEFAULT policy provides up to two HBMM master bitmaps at each site:

$ SET SHADOW/NAME=DEFAULT/POLICY=HBMM=( -
_$ (MASTER_LIST=(NYN1,NYN2,NYN3,NYN4), COUNT=2), -
_$ (MASTER_LIST=(CTN1,CTN2,CTN3,CTN4), COUNT=2), -
_$ (MASTER_LIST=(NJN1,NJN2,NJN3,NJN4), COUNT=2) )

Specifically, this policy requests master bitmaps at any two of the systems in the first master list, any two of the systems in the second master list, and any two of the systems in the third master list.

Note that you cannot accomplish this type of distribution by listing the systems in a particular order within a single MASTER_LIST. This is because the order in which the systems are specified in a master list does not affect the order in which the systems are considered when HBMM master bitmaps are created. If an event occurs that requires the creation of an HBMM master bitmap, the bitmap is created in a random order by systems that have the shadow set mounted. In the following example, the likelihood of system NYN1 getting a master bitmap is the same for either POLICY_A or POLICY_B:

$ SET SHADOW/NAME=POLICY_A/POLICY=HBMM=( -
_$ (MASTER_LIST=(NYN1,CTN1,NJN1,NYN2,CTN2,NJN2),COUNT=3) )

$ SET SHADOW/NAME=POLICY_B/POLICY=HBMM=( -
_$ (MASTER_LIST=(NJN2,CTN2,NYN2,NJN1,CTN1,NYN1),COUNT=3) )

The DISMOUNT keyword specifies the number of HBMM bitmaps to be converted to multiuse bitmaps during the manual removal of members.

If MULTIUSE is omitted, then automatic minicopy on volume processing is not enabled. As a result, no HBMM bitmap is converted to multiuse bitmap. If DISMOUNT is omitted, only a maximum of 6 HBMM bitmaps can be used as multiuse bitmaps.

For examples on the DISMOUNT keyword, see the Section 8.10.2.

On Alpha computers, you cannot use the standalone, menu-driven procedure included on the OpenVMS Alpha operating system distribution compact disc to perform BACKUP operations on shadow sets.

$ MOUNT DSA0:/SHADOW=$1$DUA10: SHADOWFACTS
%MOUNT-I-MOUNTED, SHADOWFACTS  mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA10: (DISK01) is now a
                      valid member of the shadow set

$ MOUNT DSA0:/SHADOW=$1$DUA11: SHADOWFACTS
%MOUNT-I-SHDWMEMCOPY, _$1$DUA11: (DISK02) added to the shadow
                      set with a copy operation

At this point you must wait for the copy operation to complete before dismounting the shadow set. When the copy operation is complete, messages are sent to the system console and to any operators enabled to receive them.

$ DISMOUNT DSA0:

At this point you can re-create the shadow set with one of the volumes and keep the other as a backup, or use it as a source for the backup operation.

Generally you can use the OpenVMS Backup utility (BACKUP) with shadow sets as you do with regular volumes. (See the VSI OpenVMS System Manager's Manual, Volume 1: Essentials for a description of how to back up volumes.) You can create BACKUP save sets or copies from shadow sets by using the shadow set virtual unit name instead of a physical device name as the input specifier. However, you cannot always restore to a shadow set by listing the virtual unit name as an output specifier. The main restriction to any backup restoration is that you cannot mount the target volume with the /FOREIGN qualifier. The proper procedure for a BACKUP/IMAGE restoration is described in Section 9.3.4.

BACKUP input-specifier output-specifier

$ BACKUP/RECORD  DSA2:[*…]/SINCE=BACKUP   MTA0:23DEC.BCK

This command saves all files on the shadow set DSA2 that have been created or modified since the last backup and records the current time as their new backup date.

You must take special precautions when you restore a shadow set from a BACKUP/IMAGE save set. (See the VSI OpenVMS System Manager's Manual, Volume 1: Essentials and VSI OpenVMS System Management Utilities Reference Manual, Volume 1: A-L for a description of BACKUP/IMAGE operations with physical volumes.) A BACKUP/IMAGE operation marks the target volume as more current than the other shadow set members. This designates it as the source of copy operations if you re-create the shadow set with it.

Although you can create BACKUP save sets or copies from shadow set virtual units, you cannot mount your shadow set with the /FOREIGN qualifier to allow a BACKUP/IMAGE restoration.

You should either restore to a physical disk and then re-create the shadow set with the restored disk as a shadow set member (Example 2) or, if the save operation was a copy to a compatible disk, re-create the shadow set with that disk as a member (Example 3). The target of the BACKUP/IMAGE operation becomes the source of copy operations if you re-create the shadow set with it.

Example 1

$ MOUNT DSA0:/SHADOW=($1$DUA10:, $1$DUA11:)  GHOSTVOL
%MOUNT-I-MOUNTED, GHOSTVOL     mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA10: (DISK01) is now a valid
                      member of the shadow set
%MOUNT-I-SHDWMEMSUCC, _$1$DUA11: (DISK02) is now a valid
                      member of the shadow set

$ DISMOUNT DSA0:

$ MOUNT/SYSTEM DSA0/SHADOW=$1$DUA10: GHOSTVOL
%MOUNT-I-MOUNTED, GHOSTVOL    mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA10: (DISK01) is now a valid
                      member of the shadow set

$ MOUNT $1$DUA11: GHOSTVOL
%MOUNT-W-VOLSHDWMEM, mounting a shadow set member volume
                     volume write locked
%MOUNT-I-MOUNTED, GHOSTVOL mounted on _$1$DUA11:
$ MOUNT/FOREIGN  MTA0:
%MOUNT-I-MOUNTED, …

$ BACKUP/IMAGE $1$DUA11: MTA0:SAVESET.BCK

This command produces a BACKUP/IMAGE save set from $1$DUA11 while the shadow set is on line with $1$DUA10.

Example 2

$ DISMOUNT DSA0:
$ MOUNT/FOREIGN MTA0:
%MOUNT-I-MOUNTED, …

$ MOUNT/FOREIGN/OVERRIDE=SHADOW_MEMBERSHIP $1$DUA10:
%MOUNT-I-MOUNTED, …

$ BACKUP/IMAGE MTA0:SAVESET.BCK $1$DUA10:

$ DISMOUNT/NOUNLOAD $1$DUA10:

$ MOUNT/SYSTEM DSA0/SHADOW=($1$DUA10:, $1$DUA11:) GHOSTVOL
%MOUNT-I-MOUNTED, GHOSTVOL    mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA10: (DISK01) is now a valid member of
                      the shadow set
%MOUNT-I-SHDWMEMCOPY, _$1$DUA11: (DISK02) added to the shadow set
                      with a copy operation

This command mounts the shadow set with the restored shadow set member. The output of the image backup operation has a newer generation number than other previous members of the shadow set. Therefore, $1$DUA10 (the restored volume) is the source of a copy operation when you form the shadow set.

Example 3

$ MOUNT DSA0:/SHADOW=($1$DUA10:,$1$DUA11:)  MEANDMY
%MOUNT-I-MOUNTED, MEANDMY     mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA10: (DISK03) is now a valid
                      member of the shadow set
%MOUNT-I-SHDWMEMSUCC, _$1$DUA11: (DISK04) is now a valid
                      member of the shadow set
$ MOUNT/FOREIGN $1$DUA20:
%MOUNT-I-MOUNTED, …

$ BACKUP/IMAGE/IGNORE=INTERLOCK  DSA0: $1$DUA20:

$ DISMOUNT $1$DUA20:
$ DISMOUNT DSA0:

$ MOUNT/SYSTEM DSA0/SHADOW=($1$DUA10:,$1$DUA11:,$1$DUA20:) MEANDMY
%MOUNT-I-MOUNTED, MEANDMY     mounted on _DSA0:
%MOUNT-I-SHDWMEMSUCC, _$1$DUA20: (DISK05) is now a valid
                      member of the shadow set
%MOUNT-I-SHDWMEMCOPY, _$1$DUA10: (DISK03) added to the shadow
                      set with a copy operation
%MOUNT-I-SHDWMEMCOPY, _$1$DUA11: (DISK04) added to the shadow
                      set with a copy operation

This command rebuilds the shadow set with the image backup disk as one of the shadow set members. The other former shadow set members receive copy operations.

During an unassisted shadow set merge operation, read I/O performance available to applications is reduced by two factors:

The shadow set merge operation employs a throttling mechanism to limit the impact of merge I/O on applications. The merge process is throttled by introducing a delay between merge I/Os when system load is detected.

Depending on the requirements of your application load, it may be desirable to minimize the impact of the merge I/O on your applications and allow the merge to take longer to complete; conversely, it may be desirable to make the merge complete quickly and accept the impact on applications. The following two parameters, specified with logical names, allow you to make this tradeoff for all shadow sets on your system:

The delay between merge I/O operations is computed as follows:

delay-time = (current I/O time - ideal I/O time * MERGE_DELAY_THRESHOLD/200) * 200/MERGE_DELAY_FACTOR

Increasing either parameter causes merge operations to run faster and place a heavier load on the system; conversely, decreasing them causes merge operations to run more slowly. Setting the parameters to 200 or lower slows merge operations much more gradually than in previous OpenVMS versions.

In addition to the previous two logical names which specify the parameters for all shadow sets on your system, you can specify parameters for specific shadow sets (designated by "_DSAnnnn" with logical names of the form:

You can use the same ranges of values for these parameters that you use for SHAD$MERGE_DELAY_THRESHOLD and SHAD$MERGE_DELAY_FACTOR.

The applicable logical names are sampled by the shadow copy server every 1000 I/Os, so that an in-progress copy or merge responds to a parameter change after a short delay.

There are two types of performance assists: the merge assist and the copy assist. The merge assist improves performance by using information that is maintained in controller-based write logs to merge only the data that is inconsistent across a shadow set. When a merge operation is assisted by the write logs, it is referred to as a minimerge. The copy assist reduces system resource usage and copy times by enabling a direct disk-to-disk transfer of data without going through host node memory.

Assisted merge operations are usually too short to be noticeable. Improved performance is also possible during the assisted copy operation because it consumes less CPU and interconnect resources. Although the primary purpose of the performance assists is to reduce the system resources required to perform a copy or merge operation, in some circumstances you may also observe improved read and write I/O performance.

Volume Shadowing for OpenVMS supports both assisted and unassisted shadow sets in the same OpenVMS Cluster configuration. Whenever you create a shadow set, add members to an existing shadow set, or boot a system, the shadowing software reevaluates each device in the changed configuration to determine whether it is capable of supporting either the copy assist or the minimerge. Enhanced performance is possible only as long as all shadow set members are configured on controllers that support performance assist capabilities. If any shadow set member is connected to a controller without these capabilities, the shadowing software disables the performance assist for the shadow set.

When the correct revision levels of software are installed, the copy assist and minimerge are enabled by default, and are fully managed by the shadowing software.

The copy assist and minimerge are designed to reduce the time needed to do copy and merge operations. In fact, you may notice significant time reductions on systems that have little or no user I/O occurring during the assisted copy or merge operation. Data availability is also improved because copy operations quickly make data consistent across the shadow set.

Minimerge Performance Improvements

The minimerge feature provides a significant reduction in the time needed to perform merge operations. By using controller-based write logs, it is possible to avoid the total volume scan required by earlier merge algorithms and to merge only those areas of the shadow set where write activity was known to be in progress at the time the node or nodes failed.

Unassisted merge operations often take several hours, depending on user I/O rates. Minimerge operations typically complete in a few minutes and are usually undetectable by users.

The exact time taken to complete a minimerge depends on the amount of outstanding write activity to the shadow set when the merge process is initiated, and on the number of shadow set members undergoing a minimerge simultaneously. Even under the heaviest write activity, a minimerge will complete within several minutes. Additionally, minimerge operations consume minimal compute and I/O bandwidth.

Copy Assist Performance Improvements

Copy times vary according to each configuration and generally take longer on systems supporting user I/O. Performance benefits are achieved when the source and target disks are on different HSJ internal buses.