Cluster Systems
- Operating System and Version:
- VSI OpenVMS Alpha Version 8.4-2L1 or higher
VSI OpenVMS IA-64 Version 8.4-1H1 or higher
VSI OpenVMS x86-64 Version 9.2-1 or higher
Preface
This manual describes cluster concepts, procedures, and guidelines for configuring and managing OpenVMS Cluster systems. Except where noted, the procedures and guidelines apply equally to Integrity servers and Alpha systems. This manual also includes information for providing high availability, building-block growth, and unified system management across coupled systems.
1. About VSI
VMS Software, Inc. (VSI) is an independent software company licensed by Hewlett Packard Enterprise to develop and support the OpenVMS operating system.
2. Introduction
This guide describes system management for OpenVMS Cluster systems. Although the OpenVMS Cluster software for each of the supported architectures is separately purchased, licensed, and installed, the difference between the architectures lies mainly in the hardware used. Essentially, system management for Integrity servers, Alpha computers, and x86-64 computers in an OpenVMS Cluster is identical. Exceptions are pointed out.
3. Who Should Use This Manual
This document is intended for anyone responsible for setting up and managing OpenVMS Cluster systems. To use the document as a guide to cluster management, you must have a thorough understanding of system management concepts and procedures, as described in the VSI OpenVMS System Manager's Manual.
4. How This Manual Is Organized
VSI OpenVMS Cluster Systems contains of the following chapters and appendixes.
Chapter 1, Introduction to OpenVMS Cluster System Management introduces OpenVMS Cluster systems.
Chapter 2, OpenVMS Cluster Concepts presents the software concepts integral to maintaining OpenVMS Cluster membership and integrity.
Chapter 3, OpenVMS Cluster Interconnect Configurations describes various OpenVMS Cluster configurations and the ways they are interconnected.
Chapter 4, The OpenVMS Cluster Operating Environment explains how to set up an OpenVMS Cluster system and coordinate system files.
Chapter 5, Preparing a Shared Environment explains how to set up an environment in which resources can be shared across nodes in the OpenVMS Cluster system.
Chapter 6, Cluster Storage Devices discusses disk and tape management concepts and procedures and how to use Volume Shadowing for OpenVMS to prevent data unavailability.
Chapter 7, Setting Up and Managing Cluster Queues discusses queue management concepts and procedures.
Chapter 8, Configuring an OpenVMS Cluster System explains how to build an OpenVMS Cluster system once the necessary preparations are made, and how to reconfigure and maintain the cluster.
Chapter 9, Building Large OpenVMS Cluster Systems provides guidelines for configuring and building large OpenVMS Cluster systems, booting satellite nodes, and cross-architecture booting.
Chapter 10, Maintaining an OpenVMS Cluster System describes ongoing OpenVMS Cluster system maintenance.
Appendix A, Cluster System Parameters lists and defines OpenVMS Cluster system parameters.
Appendix B, Building Common Files provides guidelines for building a cluster common user authorization file.
Appendix C, Cluster Troubleshooting provides troubleshooting information.
Appendix D, Sample Programs for LAN Control presents three sample programs for LAN control and explains how to use the Local Area OpenVMS Cluster Network Failure Analysis Program.
Appendix E, Subroutines for LAN Control describes the subroutine package used with local area OpenVMS Cluster sample programs.
Appendix F, Troubleshooting the NISCA Protocol provides techniques for troubleshooting network problems related to the NISCA transport protocol.
Appendix G, NISCA Transport Protocol Congestion Control describes how the interactions of work load distribution and network topology affect OpenVMS Cluster system performance, and discusses transmit channel selection by PEDRIVER.
5. Related Documents
This document is not a one-volume reference manual. The utilities and commands are described in detail in the VSI OpenVMS System Manager's Manual, the VSI OpenVMS System Management Utilities Reference Manual, and the VSI OpenVMS DCL Dictionary.
Guidelines for OpenVMS Cluster Configurations
VSI OpenVMS Alpha Partitioning and Galaxy Guide
VSI OpenVMS Guide to OpenVMS File Applications
VSI OpenVMS Guide to System Security
OpenVMS Alpha System Dump Analyzer Utility Manual
VMS System Dump Analyzer Utility Manual
VSI OpenVMS I/O User's Reference Manual
VSI OpenVMS License Management Utility Guide
VSI OpenVMS System Management Utilities Reference Manual
VSI OpenVMS System Manager's Manual
A Comparison of System Management on OpenVMS AXP and OpenVMS VAX
VSI OpenVMS System Services Reference Manual
VSI OpenVMS Volume Shadowing Guide
VSI OpenVMS Software Product Descriptions (available on the VSI OpenVMS website: https://vmssoftware.com/resources/documentation/)
VSI OpenVMS DECnet Network Management Utilities
VSI OpenVMS DECnet Networking Manual
The VSI DECnet–Plus (formerly known as DECnet/OSI) documentation set
The TCP/IP Services for OpenVMS documentation set
6. VSI Encourages Your Comments
You may send comments or suggestions regarding this manual or any VSI document by sending electronic mail to the following Internet address: <docinfo@vmssoftware.com>
. Users who have VSI OpenVMS support contracts through VSI can contact <support@vmssoftware.com>
for help with this product.
7. OpenVMS Documentation
The full VSI OpenVMS documentation set can be found on the VMS Software Documentation webpage at https://docs.vmssoftware.com.
8. Conventions
Convention | Meaning |
---|---|
… |
A horizontal ellipsis in examples indicates one of the
following possibilities:
|
⁝ |
A vertical ellipsis indicates the omission of items from a code example or command format; the items are omitted because they are not important to the topic being discussed. |
( ) |
In command format descriptions, parentheses indicate that you must enclose choices in parentheses if you specify more than one. |
[ ] |
In command format descriptions, brackets indicate optional choices. You can choose one or more items or no items. Do not type the brackets on the command line. However, you must include the brackets in the syntax for OpenVMS directory specifications and for a substring specification in an assignment statement. |
| |
In command format descriptions, vertical bars separate choices within brackets or braces. Within brackets, the choices are optional; within braces, at least one choice is required. Do not type the vertical bars on the command line. |
{ } |
In command format descriptions, braces indicate required choices; you must choose at least one of the items listed. Do not type the braces on the command line. |
bold text |
This typeface represents the name of an argument, an attribute, or a reason. |
italic text |
Italic text indicates important information, complete titles of manuals, or variables. Variables include information that varies in system output (Internal error number), in command lines (/PRODUCER= name), and in command parameters in text (where dd represents the predefined code for the device type). |
UPPERCASE TEXT |
Uppercase text indicates a command, the name of a routine, the name of a file, or the abbreviation for a system privilege. |
Monospace text |
Monospace type indicates code examples and interactive screen displays. In the C programming language, monospace type in text identifies the following elements:keywords, the names of independently compiled external functions and files, syntax summaries, and references to variables or identifiers introduced in an example. |
- |
A hyphen at the end of a command format description, command line, or code line indicates that the command or statement continues on the following line. |
numbers |
All numbers in text are assumed to be decimal unless otherwise noted. Nondecimal radixes—binary, octal, or hexadecimal—are explicitly indicated. |
Chapter 1. Introduction to OpenVMS Cluster System Management
“Cluster” technology was pioneered by Digital Equipment Corporation in 1983 with the VAXcluster system. The VAXcluster system was built using multiple standard VAX computing systems and the VMS operating system. The initial VAXcluster system offered the power and manageability of a centralized system and the flexibility of many physically distributed computing systems.
Through the years, the technology has evolved to support mixed-architecture cluster systems and the name changed to OpenVMS Cluster systems. Initially, OpenVMS Alpha and OpenVMS VAX systems were supported in a mixed-architecture OpenVMS Cluster system. In OpenVMS Version 8.2, cluster support was introduced for the OpenVMS Integrity server systems either in a single architecture cluster or in a mixed-architecture cluster with OpenVMS Alpha systems. VSI continues to enhance and expand OpenVMS Cluster capabilities. In OpenVMS Version 9.x, cluster support for x86-64 systems was included, both in single or mixed-architecture cluster configurations.
1.1. Overview
An OpenVMS Cluster system is a highly integrated organization of OpenVMS software, Alpha, VAX, Integrity, or x86-64 servers (or a combination of them), and storage devices that operate as a single system. The OpenVMS Cluster acts as a single virtual system, even though it is made up of many distributed systems. As members of an OpenVMS Cluster system, server systems can share processing resources, data storage, and queues under a single security and management domain, yet they can boot or shut down independently.
The distance between the computers in an OpenVMS Cluster system depends on the interconnects that you use. The computers can be located in one computer lab, on two floors of a building, between buildings on a campus, or on two different sites hundreds of miles apart.
An OpenVMS Cluster system, with computers located on two or more sites, is known as a multiple-site OpenVMS Cluster system. For more information about multiple site clusters, see the Guidelines for OpenVMS Cluster Configurations.
1.1.1. Uses
OpenVMS Cluster systems are an ideal environment for developing high-availability applications, such as transaction processing systems, servers for network client or server applications, and data-sharing applications.
1.1.2. Benefits
Benefit |
Description |
---|---|
Resource sharing |
OpenVMS Cluster software automatically synchronizes and load balances batch and print queues, storage devices, and other resources among all cluster members. |
Flexibility |
Application programmers do not have to change their application code, and users do not have to know anything about the OpenVMS Cluster environment to take advantage of common resources. |
High availability |
System designers can configure redundant hardware components to create highly available systems that eliminate or withstand single points of failure. |
Nonstop processing |
The OpenVMS operating system, which runs on each node in an OpenVMS Cluster, facilitates dynamic adjustments to changes in the configuration. |
Scalability |
Organizations can dynamically expand computing and storage resources as business needs grow or change without shutting down the system or applications running on the system. |
Performance |
An OpenVMS Cluster system can provide high performance. |
Management |
Rather than repeating the same system management operation on multiple OpenVMS systems, management tasks can be performed concurrently for one or more nodes. |
Security |
Computers in an OpenVMS Cluster share a single security database that can be accessed by all nodes in a cluster. |
Load balancing |
OpenVMS Cluster systems distribute work across cluster members based on the current load of each member. |
1.2. Hardware Components
Computers
Interconnects
Storage devices
References: Detailed OpenVMS Cluster configuration guidelines can be found in the VSI OpenVMS Cluster Software Product Description (SPD) and in Guidelines for OpenVMS Cluster Configurations.
1.2.1. Computers
Integrity server computers or workstations
Alpha computers or workstations
x86-64 computers or workstations
1.2.2. Physical Interconnects
LAN (Ethernet)
Internet Protocol (IP) (Ethernet)
MEMORY CHANNEL (node to node communications, Alpha only)
Serial Attached SCSI (SAS) (node-to-storage only, Integrity servers only)
Small Computer Systems Interface (SCSI) (node-to-storage only)
Fibre Channel (FC) (node-to-storage only)
Note
The CI, DSSI, and FDDI interconnects are supported on Alpha and VAX systems. Memory Channel interconnects are supported only on Alpha systems.
Interconnect |
Platform Support |
Comments |
---|---|---|
IP: UDP |
Integrity servers and Alpha |
Supports Fast Ethernet/Gigabit Ethernet/10 Gb Ethernet. 10 Gb Ethernet is supported on Ethernet devices only. |
Fibre Channel |
Integrity servers and Alpha |
Shared storage only |
SAS |
Integrity servers | |
SCSI |
Integrity servers and Alpha |
Limited shared storage configurations only |
LAN: Ethernet |
Integrity servers and Alpha |
The connection speed depends on NICs supported on the system and the NIC speed setting. |
MEMORY CHANNEL |
Alpha |
Node-to-node communications only |
For the most recent list of supported interconnects and speeds, see the VSI OpenVMS Cluster Software Software Product Description.
1.2.3. OpenVMS Galaxy SMCI
In addition to the physical interconnects listed in Section 1.2.2, “Physical Interconnects”, another type of interconnect, a shared memory CI (SMCI) for OpenVMS Galaxy instances, is available. SMCI supports cluster communications between Galaxy instances.
For more information about SMCI and Galaxy configurations, see the VSI OpenVMS Alpha Partitioning and Galaxy Guide.
1.2.4. Storage Devices
A shared storage device is a disk or tape that is accessed by multiple computers in the cluster. Nodes access remote disks and tapes by means of the MSCP and TMSCP server software (described in Section 1.3.1, “OpenVMS Cluster Software Functions”).
- Disks and disk drives, including:
Fibre Channel (FC) disks
SAS devices
SCSI devices
Embedded devices, such as IDE and USB devices
Digital Storage Architecture (DSA) disks
RF series integrated storage elements (ISEs)
Solid state disks
Tapes and tape drives
- Controllers and I/O servers, including the following:
Controller
Interconnect
HSG and HSV
FC
LSI 1068 and LSI Logic 1068e
SAS
HSZ
SCSI
In addition, the K.scsi HSC controller allows the connection of the StorageWorks arrays with SCSI devices on the HSC storage subsystems.
Note: HSZ controllers support many combinations of SDIs (standard disk interfaces) and STIs (standard tape interfaces) that connect disks and tapes.
For the most recent list of supported storage devices, see the VSI OpenVMS Version 8.4 Software Product Description.
In case Alpha Server Supported Options Lists or Integrity servers Supported Options Lists are needed, please contact to the HPE Support: https://support.hpe.com/.
1.3. Software Components
The OpenVMS operating system, which runs on each node in an OpenVMS Cluster, includes several software components that facilitate resource sharing and dynamic adjustments to changes in the underlying hardware configuration.
If one computer becomes unavailable, the OpenVMS Cluster system continues operating because OpenVMS is still running on the remaining computers.
1.3.1. OpenVMS Cluster Software Functions
Component | Facilitates | Function |
---|---|---|
Connection manager |
Member integrity |
Coordinates participation of computers in the cluster and maintains cluster integrity when computers join or leave the cluster. |
Distributed lock manager |
Resource synchronization |
Synchronizes operations of the distributed file system, job controller, device allocation, and other cluster facilities. If an OpenVMS Cluster computer shuts down, all locks that it holds are released so that processing can continue on the remaining computers. |
Distributed file system |
Resource sharing |
Allows all computers to share access to mass storage and file records, regardless of the type of storage device (DSA, RF, SCSI, and solid state subsystem) or its location. |
Distributed job controller |
Queuing |
Makes generic and execution queues available across the cluster. |
MSCP server |
Disk serving |
Implements the proprietary mass storage control protocol in order to make disks available to all nodes that do not have direct access to those disks. |
TMSCP server |
Tape serving |
Implements the proprietary tape mass storage control protocol in order to make tape drives available to all nodes that do not have direct access to those tape drives. |
1.4. Communications
The System Communications Architecture (SCA) defines the communications mechanisms that allow nodes in an OpenVMS Cluster system to co-operate. SCA governs the sharing of data between resources at the nodes and binds together System Applications (SYSAPs) that run on different Integrity server systems and Alpha computers.
Communications Software | Function |
---|---|
System applications (SYSAPs) |
Consists of clusterwide applications (for example, disk and tape class drivers, connection manager, and MSCP server) that use SCS software for interprocessor communication. |
System Communications Services (SCS) |
Provides basic connection management and communication services, implemented as a logical path between system applications (SYSAPs) on nodes in an OpenVMS Cluster system. |
Port drivers |
Control the communication paths between local and remote ports. |
Physical interconnects |
Consists of ports or adapters for CI, DSSI, Ethernet, ATM, FDDI, and MEMORY CHANNEL interconnects. PEDRIVER is the port driver for LAN (Ethernet) interconnect and starting with OpenVMS Version 8.4 PEDRIVER is also enabled to use TCP/IP for cluster communication. |
1.4.1. System Communications
Figure 1.1, “OpenVMS Cluster System Communications” shows the relationship between OpenVMS Cluster components.

Processes can call the $QIO system service and other system services directly from a program or indirectly using other mechanisms such as OpenVMS Record Management Services (RMS). The $QIO system service initiates all I/O requests.
- A SYSAP on one OpenVMS Cluster node communicates with a SYSAP on another node using a logical path called connection. For example, a connection manager on one node communicates with the connection manager on another node, or a disk class driver on one node communicates with the MSCP server on another node. The following SYSAPs use SCS for cluster communication:
Disk and tape class drivers
MSCP server
TMSCP server
DECnet class driver
Connection manager
SCA$TRANSPORT, which provides node-to-node communications to the intracluster communication (ICC) programming interface, available through ICC system services, and to the distributed queue manager
SCS routines provide connection setup and services to format and transfer SYSAP messages to a port driver for delivery over a specific interconnect.
Communications go through the port drivers to port drivers on other OpenVMS Cluster computers and storage controllers. A port driver manages a logical path, called a virtual circuit, between each pair of ports in an OpenVMS Cluster system. A virtual circuit provides reliable message delivery for the connections multiplexed upon it.
Starting with OpenVMS Version 8.4, cluster systems can use Transmission Control Protocol and Internet Protocol (TCP/IP) stack for cluster communication. PEDRIVER is enhanced with the capability to use TCP/IP in addition to LAN for cluster communication. For more information, see Chapter 3, OpenVMS Cluster Interconnect Configurations.
1.4.2. Application Communications
Applications running on OpenVMS Cluster systems use TCP/IP, DECnet, or ICC for application communication.
Note
The generic references to DECnet in this document mean either DECnet for OpenVMS or DECnet-Plus (formerly known as DECnet/OSI) software.
1.4.3. Cluster Alias
DECnet provides a feature known as a cluster alias. A cluster alias is a collective name for the nodes in an OpenVMS Cluster system.
Application software can use the cluster alias as the name to connect to a node in the OpenVMS Cluster. DECnet chooses the node to which the application makes a connection. The use of a cluster alias frees the application from keeping track of individual nodes in the OpenVMS Cluster system and results in design simplification, configuration flexibility, and application availability. It also provides a mechanism for load balancing by distributing incoming connections across the nodes comprising the cluster.
1.4.4. failSAFE IP
TCP/IP provides a feature known as a failSAFE IP that allows IP addresses to failover when interfaces cease functioning on a system, where multiple interfaces have been configured with the same IP address.
You can configure a standby failover target IP address that failSAFE IP assigns to multiple interfaces on a node or across the OpenVMS Cluster system. When, for example, a Network Interface Controller fails or a cable breaks or disconnects, failSAFE IP activates the standby IP address so that an alternate interface can take over to maintain the network connection. If an address is not preconfigured with a standby, then failSAFE IP removes the address from the failed interface until it recovers. When the failed interface recovers, failSAFE IP detects this and can return its IP address.
1.5. System Management
The OpenVMS Cluster system manager must manage multiple users and resources for maximum productivity and efficiency while maintaining the necessary security.
1.5.1. Ease of Management
Smaller configurations usually include only one system disk (or two for an OpenVMS Cluster configuration with both OpenVMS Alpha and OpenVMS Integrity server operating systems), regardless of the number or location of computers in the configuration.
Software must be installed only once for each operating system (Alpha or Integrity servers), and is accessible by every user and node of the OpenVMS Cluster.
Users must be added once to access the resources of the entire OpenVMS Cluster.
Several system management utilities and commands facilitate cluster management.
Figure 1.2, “Single-Point OpenVMS Cluster System Management” illustrates centralized system management.

1.5.2. Tools and Utilities
The OpenVMS operating system supports a number of utilities and tools to assist you with the management of the distributed resources in OpenVMS Cluster configurations. Proper management is essential to ensure the availability and performance of OpenVMS Cluster configurations.
Tool | Supplied or Optional | Function |
---|---|---|
Accounting | ||
VMS Accounting |
Supplied |
Tracks how resources are being used. |
Configuration and capacity planning | ||
LMF (License Management Facility) |
Supplied |
Helps the system manager determine which software products are licensed and installed on a standalone system and on each computer in an OpenVMS Cluster system. |
SYSGEN (System Generation) utility |
Supplied |
Allows you to tailor your system for a specific hardware and software configuration. Use SYSGEN to modify system parameters, load device drivers, and create additional page and swap files. |
CLUSTER_CONFIG.COM |
Supplied |
Automates the configuration or reconfiguration of an OpenVMS Cluster system and assumes the use of DECnet (for satellite booting). |
CLUSTER_CONFIG_LAN.COM |
Supplied |
Automates configuration or reconfiguration of an OpenVMS Cluster system using LANCP/LANACP (for satellite booting). |
HPE Management Agents for OpenVMS |
Supplied |
Consists of a web server for system management with management agents that allow you to look at devices on your OpenVMS systems. |
HPE Insight Manager XE |
Supplied with every HPNT server |
Centralizes system management in one system to reduce cost, improve operational efficiency and effectiveness, and minimize system down time. You can use HPE Insight Manager XE on an NT server to monitor every system in an OpenVMS Cluster system. In a configuration of heterogeneous VSI systems, you can use HPE Insight Manager XE on an NT server to monitor all systems. |
Event and fault tolerance | ||
OPCOM message routing |
Supplied |
Provides event notification. |
Operations management | ||
Clusterwide process services |
Supplied |
Allows OpenVMS system management commands, such as SHOW USERS, SHOW SYSTEM, and STOP/ID=, to operate clusterwide. |
Availability Manager |
Supplied |
From either an OpenVMS system or a Windows node, enables you to monitor one or more OpenVMS nodes on an extended LAN or wide area network (WAN). That is, the nodes for which you are collecting the information must be in the same extended LAN and there should be an interface that communicates with the collector nodes as well as the WAN analyzer. The Availability Manager collects system and process data from multiple OpenVMS nodes simultaneously, and then analyzes the data and displays the output using a native Java GUI. |
HPE WBEM Services for OpenVMS |
Supplied |
WBEM (Web-Based Enterprise Management) enables management applications to retrieve system information and request system operations wherever and whenever required. It allows customers to manage their systems consistently across multiple platforms and operating systems, providing integrated solutions that optimize your infrastructure for greater operational efficiency. |
SCACP (Systems Communications Architecture Control Program) |
Supplied |
Enables you to monitor, manage, and diagnose cluster communications and cluster interconnects. |
DNS (Distributed Name Service) |
Optional |
Configures certain network nodes as name servers that associate objects with network names. |
LATCP (Local Area Transport Control Program) |
Supplied |
Provides the function to control and obtain information from LAT port driver. |
LANCP (LAN Control Program) |
Supplied |
Allows the system manager to configure and control the LAN software on OpenVMS systems. |
NCP (Network Control Protocol) utility |
Optional |
Allows the system manager to supply and access information about the DECnet for OpenVMS (Phase IV) network from a configuration database. |
NCL (Network Control Language) utility |
Optional |
Allows the system manager to supply and access information about the DECnet–Plus network from a configuration database. |
POLYCENTER Software Installation Utility (PCSI) |
Supplied |
Provides rapid installations of software products. |
Queue Manager |
Supplied |
Uses OpenVMS Cluster generic and execution queues to feed node-specific queues across the cluster. |
Show Cluster utility |
Supplied |
Monitors activity and performance in an OpenVMS Cluster configuration, then collects and sends information about that activity to a terminal or other output device. |
SDA (System Dump Analyzer) |
Supplied |
Allows you to inspect the contents of memory saved in the dump taken at crash time or as it exists in a running system. You can use SDA interactively or in batch mode. |
SYSMAN (System Management utility) |
Supplied |
Enables device and processor control commands to take effect across an OpenVMS Cluster. |
VMSINSTAL |
Supplied |
Provides software installations. |
Performance | ||
AUTOGEN utility |
Supplied |
Optimizes system parameter settings based on usage. |
Monitor utility |
Supplied |
Provides basic performance data. |
Security | ||
Authorize utility |
Supplied |
Modifies user account profiles. |
SET ACL command |
Supplied |
Sets complex protection on many system objects. |
SET AUDIT command |
Supplied |
Facilitates tracking of sensitive system objects. |
Storage management | ||
Backup utility |
Supplied |
Allows OpenVMS Cluster system managers to create backup copies of files and directories from storage media and then restore them. This utility can be used on one node to back up data stored on disks throughout the OpenVMS Cluster system. |
Mount utility |
Supplied |
Enables a disk or tape volume for processing by one computer, a subset of OpenVMS Cluster computers, or all OpenVMS Cluster computers. |
Volume Shadowing for OpenVMS |
Optional |
Replicates disk data across multiple disks to help OpenVMS Cluster systems survive disk failures. |
1.5.3. System Management Tools from OpenVMS Partners
Schedule managers
Enable specific actions to be triggered at determined times, including repetitive and periodic activities, such as nightly backups.
Event managers
Monitor a system and report occurrences and events that may require an action or that may indicate a critical or alarming situation, such as low memory or an attempted security breaking.
Console managers
Enable a remote connection to and emulation of a system console so that system messages can be displayed and commands can be issued.
Performance managers
Monitor system performance by collecting and analyzing data to allow proper tailoring and configuration of system resources. Performance managers might also collect historical data for capacity planning.
For information about OpenVMS Partners and the tools they provide, contact to the HPE Support: https://support.hpe.com/
1.5.4. Other Configuration Aids
In addition to these utilities and partner products, several commands are available that allow the system manager to set parameters on Fibre Channel, SCSI and SAS storage subsystems to help configure and manage the system. See the appropriate hardware documentation for more information.
Chapter 2. OpenVMS Cluster Concepts
To help you understand the design and implementation of an OpenVMS Cluster system, this chapter describes its basic architecture.
2.1. OpenVMS Cluster System Architecture
Ports
System Communications Services (SCS)
System Applications (SYSAPs)
Other layered components

Note
Not all interconnects are supported on all three architectures of OpenVMS. The CI, DSSI, and FDDI interconnects are supported on Alpha and VAX systems. Memory Channel and ATM interconnects are supported only on Alpha systems.
2.1.1. Port Layer
LAN, Ethernet
Internet Protocol (IP), Ethernet
MEMORY CHANNEL
SAS
SCSI
Fibre Channel
Each interconnect is accessed by a port (also referred to as an adapter) that connects to the processor node. For example, the Fibre Channel interconnect is accessed by way of a Fibre Channel port.
2.1.2. SCS Layer
Service |
Delivery Guarantees |
Usage |
---|---|---|
Datagrams | ||
Information units that fit in 1 packet or less. |
Delivery of datagrams is not guaranteed. Datagrams can be lost, duplicated, or delivered out of order. |
Status and information messages whose loss is not critical. Applications that have their own reliability protocols such as DECnet or TCP/IP. |
Messages | ||
Information units that fit in 1 packet or less. |
Messages are guaranteed to be delivered and to arrive in order. Virtual circuit sequence numbers are used on the individual packets. |
Disk read and write requests. |
Block data transfers | ||
Copying (that is, reading or writing) any contiguous data between a local process or system virtual address space and an address on another node. Individual transfers are limited to the lesser of 232-1 bytes, or the physical memory constraints of the host. Block data is a form of remote DMA transfer. |
Delivery of block data is guaranteed. The sending and receiving ports and the port emulators cooperate in breaking the transfer into data packets and ensuring that all packets are correctly transmitted, received, and placed in the appropriate destination buffer. Block data transfers differ from messages in the size of the transfer. |
Disk subsystems and disk servers to move data associated with disk read and write requests. Fast remastering of large lock trees. Transferring large ICC messages. |
The SCS layer is implemented as a combination of hardware and software, or software only, depending upon the type of port. SCS manages connections in an OpenVMS Cluster and multiplexes messages between system applications over a common transport called a virtual circuit. A virtual circuit exists between each pair of SCS ports and a set of SCS connections that are multiplexed on that virtual circuit.
2.1.3. System Applications (SYSAPs) Layer
Connection manager
MSCP server
TMSCP server
Disk and tape class drivers
These components are described in detail later in this chapter.
2.1.4. Other Layered Components
Volume Shadowing for OpenVMS
Distributed lock manager
Process control services
Distributed file system
Record Management Services (RMS)
Distributed job controller
These components, except for volume shadowing, are described in detail later in this chapter. Volume Shadowing for OpenVMS is described in Section 6.6, “Shadowing Disks Across an OpenVMS Cluster”.
2.2. OpenVMS Cluster Software Functions
The OpenVMS Cluster software components that implement OpenVMS Cluster communication and resource-sharing functions always run on every computer in the OpenVMS Cluster. If one computer fails, the OpenVMS Cluster system continues operating, because the components still run on the remaining computers.
2.2.1. Functions
Function |
Performed By |
---|---|
Ensure that OpenVMS Cluster computers communicate with one another to enforce the rules of cluster membership |
Connection manager |
Synchronize functions performed by other OpenVMS Cluster components, OpenVMS products, and other software components |
Distributed lock manager |
Share disks and files |
Distributed file system |
Make disks available to nodes that do not have direct access |
MSCP server |
Make tapes available to nodes that do not have direct access |
TMSCP server |
Make queues available |
Distributed job controller |
2.3. Ensuring the Integrity of Cluster Membership
The connection manager ensures that computers in an OpenVMS Cluster system communicate with one another to enforce the rules of cluster membership.
Computers in an OpenVMS Cluster system share various data and system resources, such as access to disks and files. To achieve the coordination that is necessary to maintain resource integrity, the computers must maintain a clear record of cluster membership.
2.3.1. Connection Manager
Prevent partitioning (see Section 2.3.2, “Cluster Partitioning”).
Track which nodes in the OpenVMS Cluster system are active and which are not.
Deliver messages to remote nodes.
Remove nodes.
Provide a highly available message service in which other software components, such as the distributed lock manager, can synchronize access to shared resources.
2.3.2. Cluster Partitioning
A primary purpose of the connection manager is to prevent cluster partitioning, a condition in which nodes in an existing OpenVMS Cluster configuration divide into two or more independent clusters.
Cluster partitioning can result in data file corruption because the distributed lock manager cannot coordinate access to shared resources for multiple OpenVMS Cluster systems. The connection manager prevents cluster partitioning using a quorum algorithm.
2.3.3. Quorum Algorithm
The quorum algorithm is a mathematical method for determining if a majority of OpenVMS Cluster members exist so that resources can be shared across an OpenVMS Cluster system. Quorum is the number of votes that must be present for the cluster to function. Quorum is a dynamic value calculated by the connection manager to prevent cluster partitioning. The connection manager allows processing to occur only if a majority of the OpenVMS Cluster members are functioning.
2.3.4. System Parameters
Parameter | Description |
---|---|
VOTES |
Specifies a fixed number of votes that a computer
contributes toward quorum. The system manager can set the
VOTES parameters on each computer or allow the operating
system to set it to the following default values:
Each Integrity server or an Alpha computer with a nonzero value for the VOTES system parameter is considered a voting member. |
EXPECTED_VOTES |
Specifies the sum of all VOTES held by OpenVMS Cluster members. The initial value is used to derive an estimate of the correct quorum value for the cluster. The system manager must set this parameter on each active Integrity server system or an Alpha system, including satellites in the cluster. |
2.3.5. Calculating Cluster Votes
Step |
Action | |
---|---|---|
1 |
When nodes in the OpenVMS Cluster boot, the connection
manager uses the largest value for EXPECTED_VOTES of all
systems present to derive an estimated
quorum value according to the following
formula:
Estimated quorum = (EXPECTED_VOTES + 2)/2 | Rounded down | |
2 |
During a state transition (whenever a node enters or
leaves the cluster or when a quorum disk is recognized),the
connection manager dynamically computes the cluster quorum
value to be the maximum of the following:
Note: Quorum disks are discussed in Section 2.3.8, “Quorum Disk”. | |
3 |
The connection manager compares the cluster votes value to the cluster quorum value and determines what action to take based on the following conditions: | |
WHEN... |
THEN... | |
The total number of cluster votes is equal to at least the quorum value |
The OpenVMS Cluster system continues running. | |
The current number of cluster votes drops below the quorum value (because of computers leaving the cluster) |
The remaining OpenVMS Cluster members suspend all process activity and all I/O operations to cluster-accessible disks and tapes until sufficient votes are added (that is, enough computers have joined the OpenVMS Cluster) to bring the total number of votes to a value greater than or equal to quorum. |
Note: When a node leaves the OpenVMS Cluster system, the connection manager does not decrease the cluster quorum value. In fact, the connection manager never decreases the cluster quorum value; the connection manager only increases the value, unless the REMOVE NODE option was selected during shutdown. However, system managers can decrease the value according to the instructions in Section 10.11.2, “Reducing Cluster Quorum Value”.
2.3.6. Example
Consider a cluster consisting of three computers, each computer having its VOTES parameter set to 1 and its EXPECTED_VOTES parameter set to 3. The connection manager dynamically computes the cluster quorum value to be 2 (that is, (3 + 2)/2). In this example, any two of the three computers constitute a quorum and can run in the absence of the third computer. No single computer can constitute a quorum by itself. Therefore, there is no way the three OpenVMS Cluster computers can be partitioned and run as two independent clusters.
2.3.7. Sub-Cluster Selection
The subset with the highest number of votes wins, if one of the subset has more votes.
- If in case there is a tie in the number of votes:
The subset with the higher number of nodes wins.
If the number of nodes is also tied, then: OpenVMS arbitrarily, but deterministically selects one of the two subsets to "win" based on a comparison of SCS System ID values.
2.3.8. Quorum Disk
A cluster system manager can designate a disk a quorum disk. The quorum disk acts as a virtual cluster member whose purpose is to add one vote to the total cluster votes. By establishing a quorum disk, you can increase the availability of a two-node cluster; such configurations can maintain quorum in the event of failure of either the quorum disk or one node, and continue operating.
Note: Setting up a quorum disk is recommended only for OpenVMS Cluster configurations with two nodes. A quorum disk is neither necessary nor recommended for configurations with more than two nodes.
(EXPECTED VOTES + 2)/2 = (2 + 2)/2 = 2
Because there is no quorum disk, if either nonsatellite system departs from the cluster, only one vote remains and cluster quorum is lost. Activity will be blocked throughout the cluster until quorum is restored.
(EXPECTED VOTES + 2)/2 = (3 + 2)/2 = 2
Any computers that have a direct, active connection to the quorum disk or that have the potential for a direct connection should be enabled as quorum disk watchers.
Computers that cannot access the disk directly must rely on the quorum disk watchers for information about the status of votes contributed by the quorum disk.
Reference: For more information about enabling a quorum disk, see Section 8.2.4, “Adding a Quorum Disk”. Section 8.3.2, “Removing a Quorum Disk” describes removing a quorum disk.
2.3.9. Quorum Disk Watcher
Method |
Perform These Steps |
---|---|
Run the CLUSTER_CONFIG.COM procedure (described in Chapter 8, Configuring an OpenVMS Cluster System) |
Invoke the procedure and:
The procedure uses the information you provide to update the values of the DISK_QUORUM and QDSKVOTES system parameters. |
Respond YES when the OpenVMS installation procedure asks whether the cluster will contain a quorum disk (described in Chapter 4, The OpenVMS Cluster Operating Environment) |
During the installation procedure:
The procedure uses the information you provide to update the values of the DISK_QUORUM and QDSKVOTES system parameters. |
Edit the MODPARAMS or AGEN$ files (described in Chapter 8, Configuring an OpenVMS Cluster System) |
Edit the following parameters:
|
Hint: If only one quorum disk watcher has direct access to the quorum disk, then remove the disk and give its votes to the node.
2.3.10. Rules for Specifying Quorum
On all computers capable of becoming watchers, you must specify the same physical device name as a value for the DISK_QUORUM system parameter. The remaining computers (which must have a blank value for DISK_QUORUM) recognize the name specified by the first quorum disk watcher with which they communicate.
At least one quorum disk watcher must have a direct, active connection to the quorum disk.
The disk must contain a valid format file named QUORUM.DAT in the master file directory. The QUORUM.DAT file is created automatically after a system specifying a quorum disk has booted into the cluster for the first time. This file is used on subsequent reboots.
Note: The file is not created if the system parameter STARTUP_P1 is set to MIN.
To permit recovery from failure conditions, the quorum disk must be mounted by all disk watchers.
The OpenVMS Cluster can include only one quorum disk.
The quorum disk cannot be a member of a shadow set.
Hint: By increasing the quorum disk's votes to one less than the total votes from both systems (and by increasing the value of the EXPECTED_VOTES system parameter by the same amount), you can boot and run the cluster with only one node.
2.4. State Transitions
OpenVMS Cluster state transitions occur when a computer joins or leaves an OpenVMS Cluster system and when the cluster recognizes a quorum disk state change. The connection manager controls these events to ensure the preservation of data integrity throughout the cluster.
A state transition's duration and effect on users (applications) are determined by the reason for the transition, the configuration, and the applications in use.
2.4.1. Adding a Member
Every transition goes through one or more phases, depending on whether its cause is the addition of a new OpenVMS Cluster member or the failure of a current member.
Phase |
Description |
---|---|
New member detection |
Early in its boot sequence, a computer seeking membership in an OpenVMS Cluster system sends messages to current members asking to join the cluster. The first cluster member that receives the membership request acts as the new computer's advocate and proposes reconfiguring the cluster to include the computer in the cluster. While the new computer is booting, no applications are affected. Note: The connection manager will not allow a computer to join the OpenVMS Cluster system if the node's value for EXPECTED_VOTES would readjust quorum higher than calculated votes to cause the OpenVMS Cluster to suspend activity. |
Reconfiguration |
During a configuration change due to a computer being added to an OpenVMS Cluster, all current OpenVMS Cluster members must establish communications with the new computer. Once communications are established, the new computer is admitted to the cluster. In some cases, the lock database is rebuilt. |
2.4.2. Losing a Member
Cause |
Description | ||
---|---|---|---|
Failure detection |
The duration of this phase depends on the cause of the failure and on how the failure is detected. During normal cluster operation, messages sent from one computer to another are acknowledged when received. | ||
IF... |
THEN... | ||
A message is not acknowledged within a period determined by OpenVMS Cluster communications software |
The repair attempt phase begins. | ||
A cluster member is shut down or fails |
The operating system causes datagrams to be sent from the computer shutting down to the other members. These datagrams state the computer's intention to sever communications and to stop sharing resources. The failure detection and repair attempt phases are bypassed, and the reconfiguration phase begins immediately. | ||
Repair attempt |
If the virtual circuit to an OpenVMS Cluster member is broken, attempts are made to repair the path. Repair attempts continue for an interval specified by the PAPOLLINTERVAL system parameter. (System managers can adjust the value of this parameter to suit local conditions.) Thereafter, the path is considered irrevocably broken, and steps must be taken to reconfigure the OpenVMS Cluster system so that all computers can once again communicate with each other and so that computers that cannot communicate are removed from the OpenVMS Cluster. | ||
Reconfiguration |
If a cluster member is shut down or fails, the cluster must be reconfigured. One of the remaining computers acts as coordinator and exchanges messages with all other cluster members to determine an optimal cluster configuration with the most members and the most votes. This phase, during which all user (application) activity is blocked, usually lasts less than 3 seconds, although the actual time depends on the configuration. | ||
OpenVMS Cluster system recovery | Recovery includes the following stages, some of which can take place in parallel: | ||
Stage |
Action | ||
I/O completion |
When a computer is removed from the cluster, OpenVMS Cluster software ensures that all I/O operations that are started prior to the transition complete before I/O operations that are generated after the transition. This stage usually has little or no effect on applications. | ||
Recovery includes the following stages, some of which can take place in parallel: |
When a computer is removed from the cluster, OpenVMS Cluster software ensures that all I/O operations that are started prior to the transition complete before I/O operations that are generated after the transition. This stage usually has little or no effect on applications. | ||
WHEN... |
THEN... | ||
A computer leaves the OpenVMS Cluster |
A rebuild is always performed. | ||
A computer is added to the OpenVMS Cluster |
A rebuild is performed when the LOCKDIRWT system parameter is greater than 1. | ||
Caution:Setting the LOCKDIRWT system parameter to different values on the same model or type of computer can cause the distributed lock manager to use the computer with the higher value. This could cause undue resource usage on that computer. | |||
Disk mount verification |
This stage occurs only when the failure of a voting member causes quorum to be lost. To protect data integrity, all I/O activity is blocked until quorum is regained. Mount verification is the mechanism used to block I/O during this phase. | ||
Quorum disk votes validation |
If, when a computer is removed, the remaining members can determine that it has shut down or failed, the votes contributed by the quorum disk are included without delay in quorum calculations that are performed by the remaining members. However, if the quorum watcher cannot determine that the computer has shut down or failed (for example, if a console halt, power failure, or communications failure has occurred), the votes are not included for a period (in seconds) equal to four times the value of the QDSKINTERVAL system parameter. This period is sufficient to determine that the failed computer is no longer using the quorum disk. | ||
Disk rebuild |
If the transition is the result of a computer rebooting after a failure, the disks are marked as improperly dismounted. Reference: See Sections 6.5.5 and 6.5.6 for information about rebuilding disks. | ||
XFC cache change |
If the XFC cache is active on this node, a check is made to determine if there are any nodes in the cluster that do not support the XFC cache. If so, any XFC cache data must be flushed before continuing with the cluster transition. | ||
Clusterwide logical name recovery |
This stage ensures that all nodes in the cluster have matching clusterwide logical name information. | ||
Application recovery |
When you assess the effect of a state transition on application users, consider that the application recovery phase includes activities such as replaying a journal file, cleaning up recovery units, and users logging in again. |
2.5. OpenVMS Cluster Membership
Note
When using IP network for cluster communication, the remote node's IP address must be present in the SYS$SYSTEM:PE$IP_CONFIG.DAT local file.
2.5.1. Cluster Group Number
The cluster group number uniquely identifies each OpenVMS Cluster system on a LAN or IP or communicates by a common memory region (that is, communicating using SMCI). This group number must be either from 1 to 4095 or from 61440 to 65535.
Rule: If you plan to have more than one OpenVMS Cluster system on a LAN or an IP network, you must coordinate the assignment of cluster group numbers among system managers.
2.5.2. Cluster Password
The cluster password prevents an unauthorized computer using the cluster group number, from joining the cluster. The password must be from 1 to 31 characters; valid characters are letters, numbers, the dollar sign ($), and the underscore (_).
2.5.3. Location
Note
If you convert an OpenVMS Cluster that uses only the CI or DSSI interconnect to one that includes a LAN or shared memory interconnect, the SYS$COMMON:[SYSEXE]CLUSTER_AUTHORIZE.DAT file is created when you execute the CLUSTER_CONFIG.COM command procedure, as described in Chapter 8, Configuring an OpenVMS Cluster System.
Reference: For information about OpenVMS Cluster group data in the CLUSTER_AUTHORIZE.DAT file, see Sections 8.4 and 10.8.
2.5.4. Example
**** V3.4 ********************* ENTRY 343 ******************************** Logging OS 1. OpenVMS System Architecture 2. Alpha OS version XC56-BL2 Event sequence number 102. Timestamp of occurrence 16-SEP-2009 16:47:48 Time since reboot 0 Day(s) 1:04:52 Host name PERK System Model AlphaServer ES45 Model 2 Entry Type 98. Asynchronous Device Attention ---- Device Profile ---- Unit PERK$PEA0 Product Name NI-SCA Port ---- NISCA Port Data ---- Error Type and SubType x0600 Channel Error, Invalid Cluster Password Received Status x0000000000000000 Datalink Device Name EIA8: Remote Node Name CHBOSE Remote Address x000064A9000400AA Local Address x000063B4000400AA Error Count 1. Error Occurrences This Entry ----- Software Info ----- UCB$x_ERRCNT 6. Errors This Unit
2.6. Synchronizing Cluster Functions by the Distributed Lock Manager
The distributed lock manager is an OpenVMS feature for synchronizing functions required by the distributed file system, the distributed job controller, device allocation, user-written OpenVMS Cluster applications, and other OpenVMS products and software components.
The distributed lock manager uses the connection manager and SCS to communicate information between OpenVMS Cluster computers.
2.6.1. Distributed Lock Manager Functions
- Synchronizes access to shared clusterwide resources, including:
Devices
Files
Records in files
Any user-defined resources, such as databases and memory
Each resource is managed clusterwide by an OpenVMS Cluster computer.
Implements the $ENQ and $DEQ system services to provide clusterwide synchronization of access to resources by allowing the locking and unlocking of resource names.
Reference: For detailed information about system services, refer to the VSI OpenVMS System Services Reference Manual.
Queues process requests for access to a locked resource. This queuing mechanism allows processes to be put into a wait state until a particular resource is available. As a result, cooperating processes can synchronize their access to shared objects, such as files and records.
Releases all locks that an OpenVMS Cluster computer holds if the computer fails. This mechanism allows processing to continue on the remaining computers.
Supports clusterwide deadlock detection.
2.6.2. System Management of the Lock Manager
The lock manager is fully automated and usually requires no explicit system management. However, the LOCKDIRWT and LOCKRMWT system parameters can be used to adjust the distribution of activity and control of lock resource trees across the cluster.
A lock resource tree is an abstract entity on which locks can be placed. Multiple lock resource trees can exist within a cluster. For every resource tree, there is one node known as the directory node and another node known as the lock resource master node.
A lock resource master node controls a lock resource tree and is aware of all the locks on the lock resource tree. All locking operations on the lock tree must be sent to the resource master. These locks can come from any node in the cluster. All other nodes in the cluster only know about their specific locks on the tree.
Furthermore, all nodes in the cluster have many locks on many different lock resource trees, which can be mastered on different nodes. When creating a new lock resource tree, the directory node must first be queried if a resource master already exists.
The LOCKDIRWT parameter allocates a node as the directory node for a lock resource tree. The higher a node's LOCKDIRWT setting, the higher the probability that it will be the directory node for a given lock resource tree.
For most configurations, large computers and boot nodes perform optimally when LOCKDIRWT is set to 1 and satellite nodes have LOCKDIRWT set to 0. These values are set automatically by the CLUSTER_CONFIG.COM procedure. Nodes with a LOCKDIRWT of 0 will not be the directory node for any resources unless all nodes in the cluster have a LOCKDIRWT of 0.
In some circumstances, you may want to change the values of the LOCKDIRWT parameter across the cluster to control the extent to which nodes participate as directory nodes.
LOCKRMWT influences which node is chosen to remaster a lock resource tree. Because there is a performance advantage for nodes mastering a lock resource tree (as no communication is required when performing a locking operation),the lock resource manager supports remastering lock trees to other nodes in the cluster. Remastering a lock resource tree means to designate another node in the cluster as the lock resource master for that lock resource tree and to move the lock resource tree to it.
A node is eligible to be a lock resource master node if it has locks on that lock resource tree. The selection of the new lock resource master node from the eligible nodes is based on each node's LOCKRMWT system parameter setting and each node's locking activity.
Any node that has a LOCKRMWT value of 0 will attempt to remaster a lock tree to another node which has locks on that tree, as long as the other node has a LOCKRMWT greater than 0.
Nodes with a LOCKRMWT value of 10 will be given resource trees from other nodes that have a LOCKRMWT less than 10.
Otherwise, the difference in LOCKRMWT is computed between the master and the eligible node. The higher the difference, the more activity is required by the eligible node for the lock tree to move.
In most cases, maintaining the default value of 5 for LOCKRMWT is appropriate, but there may be cases where assigning some nodes a higher or lower LOCKRMWT is useful for determining which nodes master a lock tree. The LOCKRMWT parameter is dynamic, hence it can be adjusted, if necessary.
2.6.3. Large-Scale Locking Applications
The Enqueue process limit (ENQLM), which is set in the SYSUAF.DAT file and which controls the number of locks that a process can own, can be adjusted to meet the demands of large scale databases and other server applications.
Prior to OpenVMS Version 7.1, the limit was 32767. This limit was removed to enable the efficient operation of large scale databases and other server applications. A process can now own up to 16,776,959 locks, the architectural maximum. By setting ENQLM in SYSUAF.DAT to 32767 (using the Authorize utility), the lock limit is automatically extended to the maximum of 16,776,959 locks. $CREPRC can pass large quotas to the target process if it is initialized from a process with the SYSUAF Enqlm quota of 32767.
Reference: See the VSI OpenVMS Programming Concepts Manual for additional information about the distributed lock manager and resource trees. See the VSI OpenVMS System Manager's Manual for more information about Enqueue Quota.
2.7. Resource Sharing
Resource sharing in an OpenVMS Cluster system is enabled by the distributed file system, RMS, and the distributed lock manager.
2.7.1. Distributed File System
The OpenVMS Cluster distributed file system allows all computers to share mass storage and files. The distributed file system provides the same access to disks, tapes, and files across the OpenVMS Cluster that is provided on a standalone computer.
2.7.2. RMS and the Distributed Lock Manager
The distributed file system and OpenVMS Record Management Services (RMS)use the distributed lock manager to coordinate clusterwide file access. RMS files can be shared to the record level.
Connected to a supported storage subsystem
A local device that is served to the OpenVMS Cluster
All cluster-accessible devices appear as if they are connected to every computer.
2.8. Disk Availability
Locally connected disks can be served across an OpenVMS Cluster by the MSCP server.
2.8.1. MSCP Server
DSA disks local to OpenVMS Cluster members using SDI
HSG and HSV disks in an OpenVMS Cluster using mixed interconnects
SCSI and HSZ disks
SAS, LSI 1068 SAS and LSI Logic 1068e SAS disks
FC and HSG disks
Disks on boot servers and disk servers located anywhere in the OpenVMS Cluster
In conjunction with the disk class driver (DUDRIVER), the MSCP server implements the storage server portion of the MSCP protocol on a computer, allowing the computer to function as a storage controller. The MSCP protocol defines conventions for the format and timing of messages sent and received for certain families of mass storage controllers and devices designed by VSI. The MSCP server decodes and services MSCP I/O requests sent by remote cluster nodes.
Note: The MSCP server is not used by a computer to access files on locally connected disks.
2.8.2. Device Serving
Any cluster member can submit I/O requests to it.
The local computer can decode and service MSCP I/O requests sent by remote OpenVMS Cluster computers.
2.8.3. Enabling the MSCP Server
The MSCP server is controlled by the MSCP_LOAD and MSCP_SERVE_ALL system parameters. The values of these parameters are set initially by answers to questions asked during the OpenVMS installation procedure (described in Section 8.4, “Changing Computer Characteristics”), or during the CLUSTER_CONFIG.COM procedure (described in Chapter 8, Configuring an OpenVMS Cluster System).
MSCP is not loaded on satellites.
MSCP is loaded on boot server and disk server nodes.
Reference: See Section 6.3, “MSCP and TMSCP Served Disks and Tapes” for more information about setting system parameters for MSCP serving.
2.9. Tape Availability
Locally connected tapes can be served across an OpenVMS Cluster by the TMSCP server.
2.9.1. TMSCP Server
HSG and HSV tapes
SCSI tapes
SAS tapes
The TMSCP server implements the TMSCP protocol, which is used to communicate with a controller for TMSCP tapes. In conjunction with the tape class driver (TUDRIVER), the TMSCP protocol is implemented on a processor, allowing the processor to function as a storage controller.
The processor submits I/O requests to locally accessed tapes, and accepts the I/O requests from any node in the cluster. In this way, the TMSCP server makes locally connected tapes available to all nodes in the cluster. The TMSCP server can also make HSG and HSV tapes accessible to OpenVMS Cluster satellites.
2.9.2. Enabling the TMSCP Server
The TMSCP server is controlled by the TMSCP_LOAD system parameter. The value of this parameter is set initially by answers to questions asked during the OpenVMS installation procedure (described in Section 4.2.3, “Information Required”) or during the CLUSTER_CONFIG.COM procedure(described in Section 8.4, “Changing Computer Characteristics”). By default, the setting of the TMSCP_LOAD parameter does not load the TMSCP server and does not serve any tapes.
2.10. Queue Availability
Function |
Description |
---|---|
Permit users on any OpenVMS Cluster computer to submit batch and print jobs to queues that execute on any computer in the OpenVMS Cluster |
Users can submit jobs to any queue in the cluster, provided that the necessary mass storage volumes and peripheral devices are accessible to the computer on which the job executes. |
Distribute the batch and print processing work load over OpenVMS Cluster nodes |
System managers can set up generic batch and print queues that distribute processing work loads among computers. The distributed queue manager directs batch and print jobs either to the execution queue with the lowest ratio of jobs-to-queue limit or to the next available printer. |
The distributed queue manager uses the distributed lock manager to signal other computers in the OpenVMS Cluster to examine the batch and print queue jobs to be processed.
2.10.1. Controlling Queues
To control queues, you use one or several queue managers to maintain a clusterwide queue database that stores information about queues and jobs.
Reference: For detailed information about setting up OpenVMS Cluster queues, see Chapter 7, Setting Up and Managing Cluster Queues.
Chapter 3. OpenVMS Cluster Interconnect Configurations
This chapter provides an overview of various types of OpenVMS Cluster configurations and the ways they are interconnected.
VSI OpenVMS Cluster Software Product Description
Guidelines for OpenVMS Cluster Configurations
3.1. Overview
LAN (Ethernet)
Internet Protocol (IP), Ethernet
MEMORY CHANNEL (Alpha only)
SMCI (Shared memory CI) (Alpha only) in OpenVMS Galaxy configurations, as described in the VSI OpenVMS Alpha Partitioning and Galaxy Guide
SCSI (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications for limited configurations)
Fibre Channel (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications)
SAS (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications for limited configurations) (Integrity servers only)
Processing needs and available hardware resources determine how individual OpenVMS Cluster systems are configured. The configuration discussions in this chapter are based on these physical interconnects.
Note
Multihost shared storage on a SCSI interconnect, commonly known as SCSI clusters, is not supported. It is also not supported on OpenVMS Alpha systems for newer SCSI adapters. However, multihost shared storage on industry-standard Fibre Channel is supported.
Locally attached storage, on both OpenVMS Alpha systems (FC or SCSI storage) and OpenVMS Integrity server systems (Fibre Channel, SAS, or SCSI storage), can be served to any other member of the cluster.
3.2. OpenVMS Cluster Systems Interconnected by LANs
All Ethernet interconnects are industry-standard local area networks that are generally shared by a wide variety of network consumers. When OpenVMS Cluster systems are based on LAN, cluster communications are carried out by a port driver (PEDRIVER) that emulates CI port functions. When configured for Clusters Over IP (COIP), also known as IP Cluster Interconnect (IPCI), the port driver (PEDRIVER) uses TCP/IP devices as cluster transport devices. For more details see Section
3.2.1. Design
The OpenVMS Cluster software is designed to use the LAN interconnects simultaneously with the DECnet, TCP/IP, and SCS protocols. This is accomplished by allowing LAN data link software to control the hardware port. This software provides a multiplexing function so that the cluster protocols are simply another user of a shared hardware resource. See Figure 2.1, “OpenVMS Cluster System Architecture” for an illustration of this concept.
3.2.1.1. PEDRIVER Fast Path Support
Improves SMP performance scalability.
Reduces the contention for the SCS/IOLOCK8 spinlock. PEdriver uses a private port mainline spinlock to synchronize its internal operation.
Allows PEdriver to perform cluster communications processing on a secondary CPU, thus offloading the primary CPU.
Allows PEdriver to process cluster communications using a single CPU.
Reduces CPU cost by providing a Fast Path streamlined code path for DSA and served blocked data operations.
For more detailed information, see the VSI OpenVMS I/O User's Reference Manual, the VSI OpenVMS System Manager's Manual, and the VSI OpenVMS System Management Utilities Reference Manual.
3.2.2. Cluster Group Numbers and Cluster Passwords
A single LAN can support multiple LAN-based OpenVMS Cluster systems. Each OpenVMS Cluster is identified and secured by a unique cluster group number and a cluster password. Chapter 2, OpenVMS Cluster Concepts describes cluster group numbers and cluster passwords in detail.
3.2.3. Servers
Server Type | Description |
---|---|
MOP servers |
Downline load the OpenVMS boot driver to satellites by means of the Maintenance Operations Protocol (MOP). |
Disk servers |
Use MSCP server software to make their locally connected disks available to satellites over the LAN. |
Tape servers |
Use TMSCP server software to make their locally connected tapes available to satellite nodes over the LAN. |
Boot servers |
A combination of a MOP server and a disk server that serves one or more Alpha system disks. Boot and disk servers make user and application data disks available across the cluster. These servers must be the most powerful computers in the OpenVMS Cluster and must use the highest-bandwidth LAN adapters in the cluster. Boot servers must always run the MSCP server software. |
3.2.4. Satellites
Satellites are computers without a local system disk. Generally, satellites are consumers of cluster resources, although they can also provide facilities for disk serving, tape serving, and batch processing. If satellites are equipped with local disks, they can enhance performance by using such local disks for paging and swapping.
Satellites are booted remotely from a boot server (or from a MOP server and a disk server) serving the system disk. Section 3.2.5, “Satellite Booting (Alpha)” describes MOP and disk server functions during satellite booting.
3.2.5. Satellite Booting (Alpha)
Step |
Action |
Comments |
---|---|---|
1 |
Satellite requests MOP service. |
This is the original boot request that a satellite sends out across the network. Any node in the OpenVMS Cluster that has MOP service enabled and has the LAN address of the particular satellite node in its database can become the MOP server for the satellite. |
2 |
MOP server loads the Alpha system. |
The MOP server responds to an Alpha satellite boot request by downline loading the SYS$SYSTEM:APB.EXE program along with the required parameters. For Alpha computers, some of these parameters include:
|
3 |
Satellite finds additional parameters located on the system disk and root. |
OpenVMS Cluster system parameters, such as SCSSYSTEMID, SCSNODE, an NISCS_CONV_BOOT. The satellite also finds the cluster group code and password. |
4 |
Satellite executes the load program |
The program establishes an SCS connection to a disk server for the satellite system disk and loads the SYSBOOT.EXE program. |
Configuring and starting a satellite booting service for Alpha computers is described in detail in Section 4.5, “Configuring and Starting a Satellite Booting Service”.
3.2.6. Satellite Booting (Integrity servers)
Configuring and starting a satellite booting service for Integrity server systems is described in detail in Section 4.5, “Configuring and Starting a Satellite Booting Service”.
3.2.7. Configuring Multiple LAN Adapters
LAN support for multiple adapters allows PEDRIVER (the port driver for the LAN) to establish more than one channel between the local and remote cluster nodes. A channel is a network path between two nodes that is represented by a pair of LAN adapters.
3.2.7.1. System Characteristics
At boot time, all Ethernet adapters are automatically configured for local area OpenVMS Cluster use. If Cluster over IP is configured, TCP/IP is started on IP-enabled LAN adapters and configured for cluster use.
PEDRIVER automatically detects and creates a new channel between the local node and each remote cluster node for each unique pair of LAN adapters.
Channel viability is monitored continuously.
In many cases, channel failure does not interfere with node-to-node (virtual circuit)communications as long as there is at least one remaining functioning channel between the nodes.
3.2.7.2. System Requirements
The MOP server and the system disk server for a given satellite must be connected to the same extended LAN segment. (LANs can be extended using bridges that manage traffic between two or more local LANs).
All nodes must have a direct path to all other nodes. A direct path can be a bridged or a nonbridged LAN segment. When considering Cluster over IP configurations, a direct path can be via TCP/IP.
Rule: For each node, DECnet for OpenVMS (Phase IV) and MOP serving (Alpha or VAX, as appropriate) can be performed by only one adapter per extended LAN to prevent LAN address duplication.
3.2.7.3. Guidelines
The following guidelines are for configuring OpenVMS Cluster systems with multiple LAN adapters. If you configure these systems according to the guidelines, server nodes (nodes serving disks, tape, and lock traffic) can typically use some of the additional bandwidth provided by the added LAN adapters and increase the overall performance of the cluster. However, the performance increase depends on the configuration of your cluster and the applications it supports.
Connect each LAN adapter to a separate LAN segment. A LAN segment can be bridged or nonbridged. Doing this can help provide higher performance and availability in the cluster. The LAN segments can be Ethernet segments.
Distribute satellites equally among the LAN segments. Doing this can help to distribute the cluster load more equally across all of the LAN segments.
Systems providing MOP service should be distributed among the LAN segments to ensure that LAN failures do not prevent satellite booting. Systems should be bridged to multiple LAN segments for performance and availability.
For the number of LAN adapters supported per node, refer to the OpenVMS Cluster Software SPD.
3.2.8. LAN Examples
Figure 3.1, “LAN OpenVMS Cluster System with Single Server Node and System Disk” shows an OpenVMS Cluster system based on a LAN interconnect with a single OpenVMS server node and a single OpenVMS system disk.

In Figure 3.1, “LAN OpenVMS Cluster System with Single Server Node and System Disk”, the server node (and its system disk) is a single point of failure. If the server node fails, the satellite nodes cannot access any of the shared disks including the system disk. Note that some of the satellite nodes have locally connected disks. If you convert one or more of these into system disks, satellite nodes can boot from their own local system disk.
3.2.9. Fast Path for LAN Devices
Reduced contention for the SCS/IOLOCK8 spinlock. The LAN drivers now synchronize using a LAN port-specific spinlock where possible.
Offload of the primary CPU. The LAN drivers may be assigned to a secondary CPU so that I/O processing can be initiated and completed on the secondary CPU. This offloads the primary CPU and reduces cache contention between processors.
These features enhance the Fast Path functionality that already exist in LAN drivers. The enhanced functionality includes additional optimizations, preallocating of resources, and providing an optimized code path for mainline code.
For more information, see the VSI OpenVMS I/O User's Reference Manual
3.2.10. LAN Bridge Failover Process
Option |
Comments |
---|---|
Decreasing the LISTEN_TIME value allows the bridge to detect topology changes more quickly. |
If you reduce the LISTEN_TIME parameter value, you should also decrease the value for the HELLO_INTERVAL bridge parameter according to the bridge-specific guidelines. However, note that decreasing the value for the HELLO_INTERVAL parameter causes an increase in network traffic. |
Decreasing the FORWARDING_DELAY value can cause the bridge to forward packets unnecessarily to the other LAN segment. |
Unnecessary forwarding can temporarily cause more traffic on both LAN segments until the bridge software determines which LAN address is on each side of the bridge. |
Note
If you change a parameter on one LAN bridge, you should change that parameter on all bridges to ensure that selection of a new root bridge does not change he value of the parameter. The actual parameter value the bridge uses is the value specified by the root bridge.
3.2.11. Virtual LAN Support in OpenVMS
Virtual LAN (VLAN) is a mechanism for segmenting a LAN broadcast domain into smaller sections. The IEEE 802.1Q specification defines the operation and behavior of a VLAN. The OpenVMS implementation adds IEEE 802.1Q support to selected OpenVMS LAN drivers so that OpenVMS can now route VLAN tagged packets to LAN applications using a single LAN adapter.
Segment specific LAN traffic on a network for the purposes of network security or traffic containment, or both.
Use VLAN isolated networks to simplify address management.
3.2.11.1. VLAN Design
Note
DECnet-Plus and DECnet Phase IV can be configured to run over a VLAN device.

OpenVMS VLAN has been implemented through a new driver, SYS$VLANDRIVER.EXE, which provides the virtual LAN devices. Also, existing LAN drivers have been updated to handle VLAN tags. LANCP.EXE and LANACP.EXE have been updated with the ability to create and deactivate VLAN devices and to display status and configuration information.
The OpenVMS VLAN subsystem was designed with particular attention to performance. Thus, the performance cost of using VLAN support is negligible.
When configuring VLAN devices, remember that VLAN devices share the same locking mechanism as the physical LAN device. For example, running OpenVMS cluster protocol on a VLAN device along with the underlying physical LAN device does not result in increased benefit and might, in fact, hinder performance.
3.2.11.2. VLAN Support Details
All supported Gigabit and 10-Gb (Integrity servers-only) LAN devices are capable of handling VLAN traffic on Alpha and Integrity server systems.
Switch support
For VLAN configuration, the only requirement of a switch is conformance to the IEEE 802.1Q specification. The VLAN user interface to the switch is not standard; therefore, you must pay special attention when you configure as witch and especially when you configure VLANs across different switches.
LAN Failover support Figure 3.3, “LAN Failover Support” illustrates LAN Failover support.
Figure 3.3. LAN Failover Support You can create VLAN devices using a LAN Failover set as a source if all members of the set are VLAN-capable devices. However, you cannot build a Failover set using VLAN devices.
Supported capabilities
VLAN devices inherit the capability of the underlying physical LAN device, including fast path, auto-negotiation, and jumbo frame setting. If a capability needs to be modified, you must modify the underlying physical LAN device.
Restrictions
No support exists for satellite booting over a VLAN device. The OpenVMS LAN boot drivers do not include VLAN support; therefore, you cannot use a VLAN device to boot an OpenVMS system. Currently, no support exists in OpenVMS for automatic configuration of VLAN devices. You must create VLAN devices explicitly using LANCP commands.
3.3. Cluster over IP
OpenVMS Version 8.4 has been enhanced with the Cluster over IP (Internet Protocol) feature. Cluster over IP provides the ability to form clusters beyond a single LAN or VLAN segment using industry standard Internet Protocol.
System managers also have the ability to manage or monitor OpenVMS cluster that uses IP for cluster communication using SCACP management utility.
Cluster protocol (SCS also known as SCA) over LAN is provided by Port Emulator driver (PEDRIVER). PEDRIVER uses User Datagram Protocol (UDP) and IP in addition to the native cluster LAN protocol for cluster communication as shown in Figure 3.4, “Cluster Communication Design Using IP”. The datagram characteristics of UDP combined with PEDRIVER's inbuilt reliable delivery mechanism is used for transporting cluster messages which is used by SYSAP (system level application) to communicate between two cluster nodes.
Note
OpenVMS Cluster over IP and IP Cluster Interconnect (IPCI) terms are interchangeably used in the document and refers to using TCP/IP stack for cluster communication.
3.3.1. Design
PEDRIVER support for UDP protocol
TCP/IP Services boot time loading and initialization
Figure 3.4, “Cluster Communication Design Using IP” shows the cluster over IP architecture.

3.3.1.1. PEDRIVER Support for UDP
The IP UDP service has the same packet delivery characteristics as 802 LANs. PEDRIVER implements the transport layer of NISCA which has inbuilt delay probing, reliable delivery for sequenced messages (retransmission), implement datagram service and also variable buffer size for block transfers for I/O suitable for cluster traffic.
The kernel VCI (KVCI) is a kernel mode. It acts as a highly efficient interface to the VSI OpenVMS TCP/IP Services stack. It is a variant of the VCI interface, which PEdriver uses to communicate with OpenVMS LAN drivers. PEDRIVER interfaces to UDP similar to a LAN device.
Only the lowest layer of PEDRIVER is extended to support UDP. The PEDRIVER changes are transparent to PEDRIVER's upper layers.
Providing management interface ability to control and configure IP interfaces to PEDRIVER.
3.3.1.2. TCP/IP Services Boot Time Loading and Initialization
Note
Ensure that the TCP/IP software is configured before configuring cluster over IP. To ensure that network and TCP/IP is configured properly, use the PING utility and ping the node from outside the subnet.
3.3.2. Availability
The ability to create a logical LAN failover set using IP for cluster communication provides high availability systems. The nodes will be able to resume if a local LAN card fails, as it will switchover to another interface configured in the logical LAN failover set. For a complete description of creating a logical LAN failover set, see Guidelines for OpenVMS Cluster Configurations. The hardware dependency on the LAN bridge is also overcome by GbE switches or routers used for transmission and forwarding the information.
3.3.3. System Characteristics
Cluster over IP does not require any new hardware to use TCP/IP stack as interconnect.
UDP protocol is used for cluster communication.
The PEDRIVER includes delay probing technique that helps reduce latency in the IP network by selecting a path with the least latency.
The OpenVMS Cluster feature of rolling upgrades to the new version without a cluster reboot is retained.
Provides interoperability with servers running earlier versions of OpenVMS Clusters that are LAN based. Cluster over IP is available only with OpenVMS Version 8.4. Hence, if the node requires IP interconnect to be a part of the cluster, then all the nodes of the cluster must be running OpenVMS Version 8.4 and TCP/IP Services for OpenVMS, Version 5.7.
At the boot time, LAN, TCP/IP, and PEDRIVER are started sequentially.
PEDRIVER automatically detects and creates an IP channel for communication between two nodes.
Cluster over IP feature can be optionally enabled by running the CLUSTER_CONFIG_LAN.COM.
IP address used for cluster communication must be primary static address of the interface.
3.3.4. Software Requirements
OpenVMS Version 8.4 for Integrity servers or OpenVMS Alpha Version 8.4
TCP/IP Services for OpenVMS Version 5.7
Note
Ensure that the TCP/IP software is configured before configuring Cluster over IP. To ensure that network and TCP/IP is configured properly, use the PING utility and ping the node from outside the subnet.
3.3.5. Configuration Overview
IP Multicast Address
PEDRIVER uses 802 multicast for discovering cluster members in a LAN. IP multicast maps 1:1 onto the existing LAN discovery, and hence, has been selected as the preferred mechanism to discover nodes in a cluster. Every cluster using IP multicast will have one IP multicast address unique for that cluster. Multicast address is also used for keep-alive mechanism. Administratively scoped IP multicast address is used for cluster communication.
IP Unicast Address
Unicast address can be used if IP multicast is not enabled in a network. Remote node IP address must be present in the local node configuration files to allow the remote node to join the cluster. As a best practice, include all IP addresses and maintain one copy of the file throughout the cluster. $MC SCACP RELOAD can be used to refresh IP unicast list on a live system.
NISCS_USE_UDP SYSGEN Parameter
This parameter is set to enable the Cluster over IP functionality. PEDRIVER will use the UDP protocol in addition to IEEE 802.3 for cluster communication. CLUSTER_CONFIG_LAN is used to enable cluster over IP which will set this SYSGEN parameter.
UDP Port Number
Note
Standard internet practice such as firewall could be applied based on the port number that is selected for cluster.
3.3.5.1. Configuration Files
SYS$SYSTEM:PE$IP_CONFIG.DAT and SYS$SYSTEM:TCPIP$CLUSTER.DAT are the two configuration files. These files are loaded during the boot process and provide the necessary configuration details for Cluster over IP. Both these files are generated when a node is configured to be a member of the cluster and if cluster over IP is enabled during the configuration.
SYS$SYSTEM:PE$IP_CONFIG.DAT includes the optional IP multicast and IP unicast addresses of the nodes of the cluster. IP multicast messages are used for discovering a node within the same IP multicast domain. Remote nodes in a different IP multicast domain can use the IP unicast messaging technique to join the cluster. SYS$SYSTEM:PE$IP_CONFIG.DAT can be common for all the nodes of a cluster.
SYS$SYSTEM:TCPIP$CLUSTER.DAT contains the IP interface name and IP addresses on which cluster communication is enabled. It also includes the TCP/IP route information. SYS$SYSTEM:TCPIP$CLUSTER.DAT is unique for each node in a cluster.
3.3.6. Satellite Node Support
Integrity server satellite node support
The Integrity server satellite node must be in the same LAN on which the boot server resides. The Alpha satellite node must be in the same LAN as its disk server.
Alpha satellite node support
The Alpha console uses the MOP protocol for network load of satellite systems. Because the MOP protocol is non-routable, the satellite boot server or servers and all satellites booting from them must reside in the same LAN. In addition, the boot server must have at least one LAN device enabled for cluster communications to permit the Alpha satellite nodes to access the system disk.
3.3.7. High Availability Configuration using Logical LAN
The ability to create a logical LAN failover set and using IP for cluster communication with the logical LAN failover set provides high availability and can withstand NIC failure to provide high availability configuration. The nodes will be able to continue to communicate even if a local LAN card fails, as it will switchover to another interface configured in the logical LAN failover set. For a complete description of creating a logical LAN failover set and using it for Cluster over IP, see Guidelines for OpenVMS Cluster Configurations. For an example on how to create and configure a Logical LAN failover, refer to Scenario 5: Configuring an Integrity server Node Using a Logical LAN Failover set.
3.3.8. Performance Guidelines
Note
Fast path configuration is not applicable for BG devices when (Packet Processing Engine) PPE is enabled. BG device always takes the primary CPU when cluster over IP is configured and if TCP/IP stack is loaded. It is required to move the BG device to an appropriate CPU using the $SET DEVICE/PREFERRED command.
3.3.9. Example
Figure 3.5, “OpenVMS Cluster Configuration Based on IP” illustrates an OpenVMS Cluster system based on IP as interconnect. Cluster over IP enables you to connect nodes that are located across various geographical locations. IP multicast is used to locate nodes in the same domain and IP unicast is used to locate nodes in different sites or domains. Cluster over IP supports mixed-architecture, that is, a combination of Integrity server systems and Alpha systems. Lab A and Lab B have the same IP multicast address, and are connected using different LANs.
Node A and Node B are located in the same LAN and use LAN for cluster communication. However, these nodes use IP for cluster communication with all other nodes that are geographically distributed in different sites.

3.4. OpenVMS Cluster Systems Interconnected by MEMORYCHANNEL (Alpha Only)
MEMORY CHANNEL is a high-performance cluster interconnect technology for PCI-based Alpha systems. With the benefits of very low latency, high bandwidth, and direct memory access, MEMORY CHANNEL complements and extends the ability of OpenVMS Clusters to work as a single virtual system. MEMORYCHANNEL is used for node-to-node cluster communications only. You use it in combination with another interconnect, such as Fibre Channel, SCSI, CI, or DSSI, that is dedicated to storage traffic.
3.4.1. Design
PCI-to MEMORY CHANNEL adapter
Link cable (3 m or 10 feet long)
Port in a MEMORY CHANNEL hub (except for a two-node configuration in which the cable connects just two PCI adapters)
3.4.2. Examples
Figure 3.6, “Two-Node MEMORY CHANNEL OpenVMS Cluster Configuration” shows a two-node MEMORY CHANNEL cluster with shared access to Fibre Channel storage and a LAN interconnect for failover.

A three-node MEMORY CHANNEL cluster connected by a MEMORY CHANNEL hub and also by a LAN interconnect is shown in Figure 3.7, “Three-Node MEMORY CHANNEL OpenVMS Cluster Configuration”.The three nodes share access to the Fibre Channel storage. The LAN interconnect enables failover if the MEMORY CHANNEL interconnect fails.

3.5. Mixed-Interconnect OpenVMS Cluster Systems
Note
If any one node in a cluster requires IP for cluster communication, all the other members in the cluster must be enabled for IP cluster communication.
3.5.1. Availability
OpenVMS Cluster systems using a mix of interconnects provide maximum flexibility in combining CPUs, storage, and workstations into highly available configurations.
3.5.2. Examples
Figure 3.8, “OpenVMS Cluster System Using FC and Ethernet Interconnects” shows a mixed-interconnect OpenVMS Cluster system using both FC and Ethernet interconnects.
The computers based on the FC can serve HSG or HSV disks to the satellite nodes by means of MSCP server software and drivers; therefore, satellites can access the large amount of storage that is available through HSG and HSV subsystems.

3.6. Multihost SCSI OpenVMS Cluster Systems
OpenVMS Cluster systems support the SCSI as a storage interconnect. A SCSI interconnect, also called a SCSI bus, is an industry-standard interconnect that supports one or more computers, peripheral devices, and interconnecting components.
Beginning with OpenVMS Alpha Version 6.2, multiple Alpha computers using the KZPBA SCSI host-based adapter, can simultaneously access SCSI disks over a SCSI interconnect. Another interconnect, for example, a local area network, is required for host-to-host OpenVMS cluster communications. On Alpha computers, this support is limited to the KZPBA adapter. Newer SCSI host-based adapters for Alpha computers support only directly attached SCSI storage.
Beginning with OpenVMS Version 8.2-1, support is available for shared SCSI storage in a two-node OpenVMS Integrity server systems configuration using the MSA30-MI storage shelf.
Shared SCSI storage in an OpenVMS Cluster system enables computers connected to a single SCSI bus to share access to SCSI storage devices directly. This capability makes it possible to build highly available servers using shared access to SCSI storage.
3.6.1. Design for OpenVMS Alpha Configurations
Beginning with OpenVMS Alpha Version 6.2-1H3, OpenVMS Alpha supports up to three nodes on a shared SCSI bus as the storage interconnect. A quorum disk can be used on the SCSI bus to improve the availability of two-node configurations. Host-based RAID (including host-based shadowing) and the MSCP server are supported for shared SCSI storage devices.
Using the SCSI hub DWZZH-05, four nodes can be supported in a SCSI multihost OpenVMS Cluster system. In order to support four nodes, the hub's fair arbitration feature must be enabled.
For a complete description of these configurations, see Guidelines for OpenVMS Cluster Configurations.
3.6.2. Design for OpenVMS Integrity Server Shared SCSI Configurations
Maximum of two OpenVMS Integrity server systems connected to a single SCSI bus.
Maximum of four shared-SCSI buses connected to each system.
rx1600 and rx2600 family systems are supported.
A7173A HBA is the only supported HBA.
MSA30-MI storage enclosure is the only supported SCSI storage type.
Ultra320 SCSI disk family is the only supported disk family.
In Figure 3.10, “Two-Node OpenVMS Integrity server Cluster System” the SCSI IDs of 6 and 7, are required in this configuration. One of the systems must have a SCSI ID of 6 for each A7173A adapter port connected to a shared SCSI bus, instead of the factory-set default of 7. You can use the U320_SCSI pscsi.efi utility, included in the IPF Offline Diagnostics and Utilities CD, to change the SCSIID. The procedure for doing this is documented in the HP A7173APCI-X Dual Channel Ultra 320 SCSI Host Bus Adapter Installation Guide.
3.6.3. Examples
Figure 3.9, “Three-Node OpenVMS Cluster Configuration Using a Shared SCSI Interconnect” shows an OpenVMS Cluster configuration that uses a SCSI interconnect for shared access to SCSI devices. Note that another interconnect, a LAN in this example, is used for host-to-host communications.

Figure 3.10, “Two-Node OpenVMS Integrity server Cluster System” illustrates the two-node OpenVMS Integrity server configuration. Note that a second interconnect, a LAN, is required for host-to-host OpenVMS Cluster communications. (OpenVMS Cluster communications are also known as SCA (System Communications Architecture) communications).

3.7. Serial Attached SCSI (SAS) (Integrity servers Only)
OpenVMS Cluster systems support SAS as a storage interconnect. SAS is a point-to-point architecture that transfers data to and from SCSI storage devices by using serial communication (one bit at a time). SAS uses the SAS devices and differential signaling method to achieve reliable, high-speed serial communication.
SAS combines high-end features from fiber channel (such as multi-initiator support and full duplex communication) and the physical interface leveraged from SATA (for better compatibility and investment protection), with the performance, reliability and ease of use of traditional SCSI technology.
3.8. Multihost Fibre Channel OpenVMS Cluster Systems
OpenVMS Cluster systems support FC interconnect as a storage interconnect. Fibre Channel is an ANSI standard network and storage interconnect that offers many advantages over other interconnects, including high-speed transmission and long interconnect distances. A second interconnect is required for node-to-node communications.
3.8.1. Design
OpenVMS Alpha supports the Fibre Channel SAN configurations described in the latest HP Storage Works SAN Design Reference Guide and in the Data Replication Manager (DRM) user documentation. This configuration support includes multiswitch Fibre Channel fabrics, up to 500 meters of multimode fiber, and up to 100 kilometers of single-mode fiber. In addition, DRM configurations provide long-distance intersite links (ISLs) through the use of the Open Systems Gateway and wave division multiplexors. OpenVMS supports sharing of the fabric and the HSG storage with non-OpenVMS systems.
OpenVMS provides support for the number of hosts, switches, and storage controllers specified in the Storage Works documentation. In general, the number of hosts and storage controllers is limited only by the number of available fabric connections.
Host-based RAID (including host-based shadowing) and the MSCP server are supported for shared Fibre Channel storage devices. Multipath support is available for these configurations.
For a complete description of these configurations, see Guidelines for OpenVMS Cluster Configurations.
Chapter 4. The OpenVMS Cluster Operating Environment
This chapter describes how to prepare the OpenVMS Cluster operating environment.
4.1. Preparing the Operating Environment
Task |
Section |
---|---|
Check all hardware connections to computer, interconnects, and devices. |
Described in the appropriate hardware documentation. |
Verify that all microcode and hardware is set to the correct revision levels. |
Contact your support representative. |
Install the OpenVMS operating system. | |
Install all software licenses, including OpenVMS Cluster licenses. | |
Install layered products. | |
Configure and start LANCP or DECnet for satellite booting |
Section 4.5, “Configuring and Starting a Satellite Booting Service” |
4.2. Installing the OpenVMS Operating System
Install the OpenVMS Integrity servers operating system on each Integrity system disk
Install the OpenVMS Alpha operating system on each Alpha system disk
Install the OpenVMS x86-64 operating system on each x86-64 system disk
Note
Mixed architecture clusters of OpenVMS Integrity servers, Alpha systems, and x86-64 systems are supported.
4.2.1. System Disks
A system disk is one of the few resources that cannot be shared between Integrity and Alpha systems.
Note
An OpenVMS Cluster running both implementations of DECnet requires a system disk for DECnet for OpenVMS (Phase IV) and another system disk for DECnet-Plus (Phase V). For more information, see the DECnet-Plus documentation.
4.2.2. Where to Install
IF you want the cluster to have... |
THEN perform the installation or upgrade... |
---|---|
One common system disk for all computer members |
Once on the cluster common system disk. |
A combination of one or more common system disks and one or more local (individual) system disks |
|
Note: If your cluster includes multiple common system disks, you must later coordinate system files to define the cluster operating environment, as described in Chapter 5, Preparing a Shared Environment. Reference: See Section 8.5, “Creating a Duplicate System Disk” for information about creating a duplicate system disk. |
Example: If your OpenVMS Cluster consists of 10 computers, four of which boot from a common Integrity server system disk, two of which boot from a second common Integrity system disk, two of which boot from a common Alpha system disk, and two of which boot from their own local system disk, you need to perform an installation five times.
4.2.3. Information Required
Table 4.1, “Information Required to Perform an Installation” table lists the questions that the OpenVMS operating system installation procedure prompts you with and describes how certain system parameters are affected by responses you provide. You will notice that two of the prompts vary, depending on whether the node is running DECnet. The table also provides an example of an installation procedure that is taking place on a node named JUPITR.
Important: Be sure you determine answers to the questions before you begin the installation.
Note about versions: Refer to the appropriate OpenVMS Release Notes document for the required version numbers of hardware and firmware. When mixing versions of the operating system in an OpenVMS Cluster, check the release notes for information about compatibility.
Prompt | Response | Parameter | ||
---|---|---|---|---|
Will this node be a cluster member (Y/N)? |
WHEN you respond... |
AND... |
THEN the VAXcluster parameter is set to... |
VAXCLUSTER |
N |
CI and DSSI hardware is not present |
0 — Node will not participate in the OpenVMS Cluster. | ||
N |
CI and DSSI hardware is present |
1 — Node will automatically participate in the OpenVMS Cluster in the presence of CI or DSSI hardware. | ||
Y |
2 — Node will participate in the OpenVMS Cluster. | |||
What is the node's DECnet node name? |
If the node is running DECnet, this prompt, the following prompt, and the SCSSYSTEMID prompt are displayed. Enter the DECnet node name or the DECnet–Plus node synonym (for example, JUPITR). If a node synonym is not defined, SCSNODE can be any name from 1 to 6 alphanumeric characters in length. The name cannot include dollar signs ($) or underscores (_). |
SCSNODE | ||
What is the node's DECnet node address? |
Enter the DECnet node address (for example, a valid address might be 2.211). If an address has not been assigned, enter 0 now and enter a valid address when you start DECnet (discussed later in this chapter). For DECnet–Plus, this question is asked when nodes are configured with a Phase IV compatible address. If a Phase IV compatible address is not configured, then the SCSSYSTEMID system parameter can be set to any value. |
SCSSYSTEMID | ||
What is the node's SCS node name? |
If the node is not running DECnet, this prompt and the following prompt are displayed in place of the two previous prompts. Enter a name of 1 to 6 alphanumeric characters that uniquely names this node. At least 1 character must be a letter. The name cannot include dollar signs ($) or underscores (_). |
SCSNODE | ||
What is the node's SCSSYSTEMID number? |
This number must be unique within this cluster. SCSSYSTEMID is the low-order 32 bits of the 48-bit system identification number. If the node is running DECnet for OpenVMS, calculate the
value from the DECnet address using the following formula:
Example: If the DECnet
address is 2.211, calculate the value as
follows:
SCSSYSTEMID = (2 * 1024) + 211 = 2259 |
SCSSYSTEMID | ||
Will the Ethernet be used for cluster communications (Y/N)? |
IF you respond... |
THEN the NISCS_LOAD_PEA0 parameter is set to... |
NISCS_LOAD_PEA0 | |
N |
0 — PEDRIVER is not loaded?; cluster communications does not use Ethernet or FDDI. | |||
Y |
1 — Loads PEDRIVER to enable cluster communications over Ethernet or FDDI. | |||
Will the IP interconnect be used for cluster communications (Y/N)? |
IF you respond... |
THEN the NISCS_USE_UDP parameter is set to... |
NISCS_USE_UDP | |
N |
0 — Cluster over IP is disabled and uses the LAN interconnect for cluster communication | |||
Y |
1 — Cluster over IP is enabled and communicates using the TCP/IP stack. During the boot process, the TCP/IP driver and then the PEDRIVER authorization information is loaded for cluster communication. The hello packets are transmitted using IP multicast and unicast. | |||
Enter this cluster's group number: |
Enter a number in the range of 1 to 4095 or 61440 to 65535 (see Section 2.5, “OpenVMS Cluster Membership”). This value is stored in the CLUSTER_AUTHORIZE.DAT file in the SYS$COMMON:[SYSEXE] directory. |
Not applicable | ||
Enter this cluster's password: |
Enter the cluster password. The password must be from 1 to 31 alphanumeric characters in length and can include dollar signs ($) and underscores (_) (see Section 2.5, “OpenVMS Cluster Membership”).This value is stored in scrambled form in the CLUSTER_AUTHORIZE.DAT file in the SYS$COMMON:[SYSEXE] directory. |
Not applicable | ||
Reenter this cluster's password for verification: |
Reenter the password. |
Not applicable | ||
Will JUPITR be a disk server (Y/N)? |
IF you respond... |
THEN the MSCP_LOAD parameter is set to... |
MSCP_LOAD | |
N |
0 — The MSCP server will not be loaded. This is the correct setting for configurations in which all OpenVMS Cluster nodes can directly access all shared storage and do not require LAN failover. | |||
Y |
1 — Loads the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter, using the default CPU load capacity. | |||
Will JUPITR serve HSC or RF disks (Y/N)? |
IF you respond... |
THEN the MSCP_SERVE_ALL parameter is set to... |
MSCP_SERVE_ALL | |
Y |
1 — Serves all available disks. | |||
N |
2 — Serves only locally connected (not HSC, HSJ, or RF) disks. | |||
Enter a value for JUPITR's ALLOCLASS parameter:? |
The value is dependent on the system configuration:
|
ALLOCLASS | ||
Does this cluster contain a quorum disk [N]? |
Enter Y or N, depending on your configuration. If you enter Y, the procedure prompts for the name of the quorum disk. Enter the device name of the quorum disk. (Quorum disks are discussed in Chapter 2, OpenVMS Cluster Concepts). |
DISK_QUORUM |
4.3. Installing Software Licenses
While rebooting at the end of the installation procedure, the system displays messages warning that you must install the operating system software and the OpenVMS Cluster software license. The OpenVMS Cluster software supports the OpenVMS License Management Facility (LMF). License units for clustered systems are allocated on an unlimited system-use basis.
4.3.1. Guidelines
Be sure to install all OpenVMS Cluster licenses and all licenses for layered products and DECnet as soon as the system is available. Procedures for installing licenses are described in the release notes distributed with the software kit and in the VSI OpenVMS License Management Utility Guide. Additional licensing information is described in the respective SPDs.
Install an OpenVMS Cluster Software for Alpha license for each Alpha processor in the OpenVMS Cluster.
Install an OpenVMS Cluster Software for Integrity server system license for each Integrity server processor in the OpenVMS Cluster.
Install or upgrade licenses for layered products that runs on all nodes in an OpenVMS Cluster system.
OpenVMS Product Authorization Keys (PAKs) that have the Alpha option can be loaded and used only on Alpha processors. PAKs that have the Integrity servers option can be loaded and used only on Integrity server processors. However, PAKs can be located in a license database (LDB) that is shared by all processors(Integrity servers and Alpha).
PAK types, such as Activity PAKs (also known as concurrent or n-user PAKs)and Personal Use PAKs (identified by the RESERVE_UNITS option) work on Alpha systems.
PAK types, such as PCL PAKs (per core licensing) are only supported on Integrity servers.
License management commands can be issued from every node in the cluster.
4.4. Installing Layered Products
By installing layered products before other nodes are added to the OpenVMS Cluster, the software is installed automatically on new members when they are added to the OpenVMS Cluster system.
Note: For clusters with multiple system disks (Integrity servers or Alpha) you must perform a separate installation for each system disk.
4.4.1. Procedure
Phase |
Action |
---|---|
Before installation |
Perform one or more of the following steps, as necessary
for your system.
|
Installation |
Refer to the appropriate layered-product documentation for product-specific installation information. Perform the installation once for each system disk. |
After installation |
Perform one or more of the following steps, as necessary
for your system.
|
4.5. Configuring and Starting a Satellite Booting Service
After you have installed the operating system and the required licenses on the first OpenVMS Cluster computer, you can configure and start a satellite booting service. You can use the LANCP utility, or DECnet software, or both.
VSI recommends LANCP for booting OpenVMS Cluster satellites. LANCP has shipped with the OpenVMS operating system since Version 6.2. It provides a general-purpose MOP booting service that can be used for booting satellites into an OpenVMS Cluster. (LANCP can service all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals, and can be used to customize your LAN environment).
Note
Replace DECnet with LANCP for satellite booting (MOP downline load service) using LAN$POPULATE.COM.
Migrate from DECnet Phase IV to DECnet-Plus.
There are two cluster configuration command procedures, CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. CLUSTER_CONFIG_LAN.COM uses LANCP to provide MOP services to boot satellites; CLUSTER_CONFIG.COM uses DECnet for the same purpose.
Applications you will be running on your cluster
DECnet task-to-task communications is a method commonly used for communication between programs that run on different nodes in a cluster or a network. If you are running a program with that dependency, you need to run DECnet. If you are not running any programs with that dependency, you do not need to run DECnet.
Limiting applications that require DECnet to certain nodes in your cluster
If you are running applications that require DECnet task-to-task communications, you can run those applications on a subset of the nodes in your cluster and restrict DECnet usage to those nodes. You can use LANCP software on the remaining nodes and use a different network, such as TCP/IP Services for OpenVMS, for other network services.
Managing two types of software for the same purpose
If you are already using DECnet for booting satellites, you may not want to introduce another type of software for that purpose. Introducing any new software requires time to learn and manage it.
- LANCP MOP services can coexist with DECnet MOP services in an OpenVMS Cluster in the following ways:
Running on different systems
For example, DECnet MOP service is enabled on some of the systems on the LAN and LAN MOP is enabled on other systems.
Running on different LAN devices on the same system
For example, DECnet MOP service is enabled on a subset of the available LAN devices on the system and LAN MOP is enabled on the remainder.
Running on the same LAN device on the same system but targeting a different set of nodes for service
For example, both DECnet MOP and LAN MOP are enabled but LAN MOP has limited the nodes to which it will respond. This allows DECnet MOP to respond to the remaining nodes.
Instructions for configuring both LANCP and DECnet are provided in this section.
4.5.1. Configuring and Starting the LANCP Utility
You can use the LAN Control Program (LANCP) utility to configure a local area network (LAN). You can also use the LANCP utility, in place of DECnet or in addition to DECnet, to provide support for booting satellites in an OpenVMS Cluster and for servicing all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals.
Reference: For more information about using the LANCP utility to configure a LAN, see the VSI OpenVMS System Manager's Manual and the VSI OpenVMS System Management Utilities Reference Manual.
4.5.2. Booting Satellite Nodes with LANCP
The LANCP utility provides a general-purpose MOP booting service that can be used for booting satellites into an OpenVMS Cluster. It can also be used to service all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals. To use LANCP for this purpose, all OpenVMS Cluster nodes must be running OpenVMS Version 6.2 or higher.
The CLUSTER_CONFIG_LAN.COM cluster configuration command procedure uses LANCP in place of DECnet to provide MOP services to boot satellites.
Replace DECnet with LANCP for satellite booting (MOP downline load service), using LAN$POPULATE.COM.
Migrate from DECnet for OpenVMS to DECnet–Plus.
4.5.3. Data Files Used by LANCP
SYS$SYSTEM:LAN$DEVICE_DATABASE.DAT
This file maintains information about devices on the local node. By default, the file is created in SYS$SPECIFIC:[SYSEXE], and the system looks for the file in that location. However, you can modify the file name or location for this file by redefining the systemwide logical name LAN$DEVICE_DATABASE.
SYS$SYSTEM:LAN$NODE_DATABASE.DAT
This file contains information about the nodes for which LANCP will supply boot service. This file must be shared among all nodes in the OpenVMS Cluster, including Integrity servers, Alpha, and VAX systems. By default, the file is created in SYS$COMMON:[SYSEXE], and the system looks for the file in that location. However, you can modify the file name or location for this file by redefining the systemwide logical name LAN$NODE_DATABASE.
4.5.4. Using LAN MOP Services in New Installations
Add the startup command for LANCP.
You should start up LANCP as part of your system startup procedure. To do this, remove the comment from the line in SYS$MANAGER:SYSTARTUP_VMS.COM that runs the LAN$STARTUP command procedure. If your OpenVMS Cluster system will have more than one system disk, see Section 4.5.3, “Data Files Used by LANCP ” for a description of logicals that can be defined for locating LANCP configuration files.$ @SYS$STARTUP:LAN$STARTUP
You should now either reboot the system or invoke the preceding command procedure from the system manager's account to start LANCP.
Follow the steps in Chapter 8, Configuring an OpenVMS Cluster System for configuring an OpenVMS Cluster system and adding satellites. Use the CLUSTER_CONFIG_LAN.COM command procedure instead of CLUSTER_CONFIG.COM. If you invoke CLUSTER_CONFIG.COM, it gives you the option to switch to running CLUSTER_CONFIG_LAN.COM if the LANCP process has been started.
4.5.5. Using LAN MOP Services in Existing Installations
Redefine the LANCP database logical names.
This step is optional. If you want to move the data files used by LANCP, LAN$DEVICE_DATABASE and LAN$NODE_DATABASE, off the system disk, redefine their systemwide logical names. Add the definitions to the system startup files.
Use LANCP to create the LAN$DEVICE_DATABASE
The permanent LAN$DEVICE_DATABASE is created when you issue the first LANCP DEVICE command. To create the database and get a list of available devices, enter the following commands:$ MCR LANCP LANCP> LIST DEVICE /MOPDLL %LANCP-I-FNFDEV, File not found, LAN$DEVICE_DATABASE %LANACP-I-CREATDEV, Created LAN$DEVICE_DATABASE file Device Listing, permanent database: --- MOP Downline Load Service Characteristics --- Device State Access Mode Client Data Size ------ ----- ----------- ------ --------- ESA0 Disabled NoExlusive NoKnownClientsOnly 246 bytes FCA0 Disabled NoExlusive NoKnownClientsOnly 246 bytes
Use LANCP to enable LAN devices for MOP booting.
By default, the LAN devices have MOP booting capability disabled. Determine the LAN devices for which you want to enable MOP booting. Then use the DEFINE command in the LANCP utility to enable these devices to service MOP boot requests in the permanent database, as shown in the following example:LANCP> DEFINE DEVICE ESA0:/MOP=ENABLE
Run LAN$POPULATE.COM (found in SYS$EXAMPLES) to obtain MOP booting information and to produce LAN$DEFINE and LAN$DECNET_MOP_CLEANUP, which are site specific.
LAN$POPULATE extracts all MOP booting information from a DECnet Phase IVNETNODE_REMOTE.DAT file or from the output of the DECnet–Plus NCL command SHOW MOP CLIENT * ALL.
For DECnet Phase IV sites, the LAN$POPULATE procedure scans all DECnet areas (1–63) by default. If you MOP boot systems from only a single or a few DECnet areas, you can cause the LAN$POPULATE procedure to operate on a single area at a time by providing the area number as the P1 parameter to the procedure, as shown in the following example (including log):$ @SYS$EXAMPLES:LAN$POPULATE 15 LAN$POPULATE - V1.0 Do you want help (Y/N) <N>: LAN$DEFINE.COM has been successfully created. To apply the node definitions to the LANCP permanent database, invoke the created LAN$DEFINE.COM command procedure. VSI recommends that you review LAN$DEFINE.COM and remove any obsolete entries prior to executing this command procedure. A total of 2 MOP definitions were entered into LAN$DEFINE.COM
Run LAN$DEFINE.COM to populate LAN$NODE_DATABASE.
LAN$DEFINE populates the LANCP downline loading information into the LAN node database, SYS$COMMON:[SYSEVE]LAN$NODE_DATABASE.DAT file. VSI recommends that you review LAN$DEFINE.COM and remove any obsolete entries before executing it.
In the following sequence, the LAN$DEFINE.COM procedure that was just created is displayed on the screen and then executed:$ TYPE LAN$DEFINE.COM $ ! $ ! This file was generated by LAN$POPULATE.COM on 16-DEC-1996 09:20:31 $ ! on node CLU21. $ ! $ ! Only DECnet Area 15 was scanned. $ ! $ MCR LANCP Define Node PORK /Address=08-00-2B-39-82-85 /File=APB.EXE - /Root=$21$DKA300:<SYS11.> /Boot_type=Alpha_Satellite Define Node JYPIG /Address=08-00-2B-A2-1F-81 /File=APB.EXE - /Root=$21$DKA300:<SYS10.> /Boot_type=Alpha_Satellite EXIT $ @LAN$DEFINE %LANCP-I-FNFNOD, File not found, LAN$NODE_DATABASE -LANCP-I-CREATNOD, Created LAN$NODE_DATABASE file $
The following example shows a LAN$DEFINE.COM command procedure that was generated by LAN$POPULATE for migration from DECnet–Plus to LANCP.$ ! LAN$DEFINE.COM - LAN MOP Client Setup $ ! $ ! This file was generated by LAN$POPULATE.COM at 8-DEC-1996 14:28:43.31 $ ! on node BIGBOX. $ ! $ SET NOON $ WRITE SYS$OUTPUT "Setting up MOP DLL clients in LANCP... $ MCR LANCP SET NODE SLIDER /ADDRESS=08-00-2B-12-D8-72/ROOT=BIGBOX$DKB0:<SYS10.>/BOOT_TYP E=VAX_satellite/FILE=NISCS_LOAD.EXE DEFINE NODE SLIDER /ADDRESS=08-00-2B-12-D8-72/ROOT=BIGBOX$DKB0:<SYS10.>/BOOT_TYP E=VAX_satellite/FILE=NISCS_LOAD.EXE EXIT $ ! $ WRITE SYS$OUTPUT "DECnet Phase V to LAN MOPDLL client migration complete!" $ EXIT
Run LAN$DECNET_MOP_CLEANUP.COM.
You can use LAN$DECNET_MOP_CLEANUP.COM to remove the clients' MOP downline loading information from the DECnet database. VSI recommends that you review LAN$DECNET_MOP_CLEANUP.COM and remove any obsolete entries before executing it.
The following example shows a LAN$DECNET_MOP_CLEANUP.COM command procedure that was generated by LAN$POPULATE for migration from DECnet–Plus to LANCP.
Note: When migrating from DECnet–Plus, additional cleanup is necessary. You must edit your NCL scripts (*.NCL) manually.$ ! LAN$DECNET_MOP_CLEANUP.COM - DECnet MOP Client Cleanup $ ! $ ! This file was generated by LAN$POPULATE.COM at 8-DEC-1995 14:28:43.47 $ ! on node BIGBOX. $ ! $ SET NOON $ WRITE SYS$OUTPUT "Removing MOP DLL clients from DECnet database..." $ MCR NCL DELETE NODE 0 MOP CLIENT SLIDER EXIT $ ! $ WRITE SYS$OUTPUT "DECnet Phase V MOPDLL client cleanup complete!" $ EXIT
Start LANCP.
To start LANCP, execute the startup command procedure as follows:$ @SYS$STARTUP:LAN$STARTUP %RUN-S-PROC_ID, identification of created process is 2920009B $
You should start up LANCP for all boot nodes as part of your system startup procedure. To do this, include the following line in your site-specific startup file (SYS$MANAGER:SYSTARTUP_VMS.COM):$ @SYS$STARTUP:LAN$STARTUP
If you have defined logicals for either LAN$DEVICE_DATABASE or LAN$NODE_DATABASE, be sure that these are defined in your startup files prior to starting up LANCP.
Disable DECnet MOP booting.
If you use LANCP for satellite booting, you may no longer need DECnet to handle MOP requests. If this is the case for your site, you can turn off this capability with the appropriate NCP command (DECnet for OpenVMS) or NCL commands (DECnet–Plus).
For more information about the LANCP utility, see the VSI OpenVMS System Manager's Manual and the VSI OpenVMS System Management Utilities Reference Manual.
4.5.6. Configuring DECnet
The process of configuring the DECnet network typically entails several operations, as shown in Table 4.3, “Procedure for Configuring the DECnet Network”.An OpenVMS Cluster running both implementations of DECnet requires a system disk for DECnet for OpenVMS (Phase IV) and another system disk for DECnet–Plus (Phase V).
Note: DECnet for OpenVMS implements Phase IV of Digital Network Architecture (DNA). DECnet–Plus implements Phase V of DNA. The following discussions are specific to the DECnet for OpenVMS product.
Step | Action |
---|---|
1 |
Log in as system manager and execute the NETCONFIG.COM command procedure as shown. Enter information about your node when prompted. Note that DECnet–Plus nodes execute the NET$CONFIGURE.COM command procedure. Reference: See the DECnet for OpenVMS or the DECnet–Plus documentation, as appropriate, for examples of these procedures. |
2 |
When a node uses multiple LAN adapter connections to the same LAN and also uses DECnet for communications, you must disable DECnet use of all but one of the LAN devices. To do this, remove all but one of the lines and circuits associated with the adapters connected to the same LAN or extended LAN from the DECnet configuration database after the NETCONFIG.COM procedure is run. For example, issue the following commands to invoke NCP
and disable DECnet use of the LAN device
XQB0:
$ RUN SYS$SYSTEM:NCP NCP> PURGE CIRCUIT QNA-1 ALL NCP> DEFINE CIRCUIT QNA-1 STA OFF NCP> EXIT References: See Guidelines for OpenVMS Cluster Configurations for more information about distributing connections to LAN segments in OpenVMS Cluster configurations. See the DECnet–Plus documentation for information about removing routing circuits associated with all but one LAN adapter. (Note that the LAN adapter issue is not a problem if the DECnet–Plus node uses extended addressing and does not have any Phase IV compatible addressing in use on any of the routing circuits). |
3 |
Make remote node data available clusterwide. NETCONFIG.COM creates in the SYS$SPECIFIC:[SYSEXE] directory the permanent remote-node database file NETNODE_REMOTE.DAT, in which remote-node data is maintained. To make this data available throughout the OpenVMS Cluster, you move the file to the SYS$COMMON:[SYSEXE] directory. Example: Enter the
following commands to make DECnet information available
clusterwide:
$ RENAME SYS$SPECIFIC:[SYSEXE]NETNODE_REMOTE.DAT SYS$COMMON:[SYSEXE]NETNODE_REMOTE.DAT If your configuration includes multiple system disks, you
can set up a common NETNODE_REMOTE.DAT file automatically by
using the following command in
SYLOGICALS.COM:
$ DEFINE/SYSTEM/EXE NETNODE_REMOTE
ddcu:[directory]NETNODE_REMOTE.DAT Notes: VSI recommends that you set up a common NETOBJECT.DAT file clusterwide in the same manner. DECdns is used by DECnet–Plus nodes to manage node data (the namespace). For DECnet–Plus, Session Control Applications replace objects. |
4 |
Designate and enable router nodes to support the use of a cluster alias. At least one node participating in a cluster alias must be configured as a level 1 router. On Integrity servers and Alpha systems, you might need to enable level 1 routing manually because the NETCONFIG.COM procedure does not prompt you with the routing question. Depending on whether the configuration includes a combination of Integrity sever nodes and Alpha nodes, you must enable level 1 routing manually (see the example below) on one of the Alpha nodes. Example: On Alpha
systems, if you need to enable level 1 routing on Alpha
node, invoke the NCP utility to do so. For
example:
$ RUN SYS$SYSTEM:NCP NCP> DEFINE EXECUTOR TYPE ROUTING IV Note: On Integrity servers and Alpha systems, level 1 routing is supported to enable cluster alias operations only. |
5 |
Optionally, define a cluster alias. If you want to define a cluster alias, invoke the NCP utility to do so. The information you specify using these commands is entered in the DECnet permanent executor database and takes effect when you start the network.
Example: The following NCP
commands establish SOLAR as an
alias:
$ RUN SYS$SYSTEM:NCP NCP> DEFINE NODE 2.1 NAME SOLAR NCP> DEFINE EXECUTOR ALIAS NODE SOLAR NCP> EXIT $ Reference: Section 4.5.8, “What is Cluster Alias?” describes the cluster alias. Section 4.5.9, “Enabling Alias Operations” describes how to enable alias operations for other computers. See the DECnet–Plus documentation for information about setting up a cluster alias on DECnet–Plus nodes. Note: DECnet for OpenVMS nodes and DECnet–Plus nodes cannot share a cluster alias. |
4.5.7. Starting DECnet
If you are using DECnet–Plus, a separate step is not required to start the network. DECnet–Plus starts automatically on the next reboot after the node has been configured using the NET$CONFIGURE.COM procedure.
$ @SYS$MANAGER:STARTNET.COM
To ensure that the network is started each time an OpenVMS Cluster computer boots, add that command line to the appropriate startup command file or files. (Startup command files are discussed in Section 5.5, “Coordinating Startup Command Procedures”).
4.5.8. What is Cluster Alias?
The cluster alias acts as a single network node identifier for an OpenVMS Cluster system. When enabled, the cluster alias makes all the OpenVMS Cluster nodes appear to be one node from the point of view of the rest of the network.
Computers in the cluster can use the alias for communications with other computers in a DECnet network. For example, networked applications that use the services of an OpenVMS Cluster should use an alias name. Doing so ensures that the remote access will be successful when at least one OpenVMS Cluster member is available to process the client program's requests.
DECnet for OpenVMS (Phase IV) allows a maximum of 64 OpenVMS Cluster computers to participate in a cluster alias. If your cluster includes more than 64 computers, you must determine which 64 should participate in the alias and then define the alias on those computers.
At least one of the OpenVMS Cluster nodes that uses the alias node identifier must have level 1 routing enabled.On Integrity servers and Alpha nodes, routing between multiple circuits is not supported. However, routing is supported to allow cluster alias operations. Level 1 routing is supported only for enabling the use of a cluster alias. The DVNETEXT PAK must be used to enable this limited function.
On Integrity servers, Alpha, and VAX systems, all cluster nodes sharing the same alias node address must be in the same area.
DECnet–Plus allows a maximum of 96 OpenVMS Cluster computers to participate in the cluster alias.
DECnet–Plus does not require that a cluster member be a routing node, but an adjacent Phase V router is required to use a cluster alias for DECnet–Plus systems.
A single cluster alias can include nodes running either DECnet for OpenVMS or DECnet–Plus, but not both.
4.5.9. Enabling Alias Operations
- Log in as system manager and invoke the SYSMAN utility. For example:
$ RUN SYS$SYSTEM:SYSMAN SYSMAN>
- At the SYSMAN> prompt, enter the following commands:
SYSMAN> SET ENVIRONMENT/CLUSTER %SYSMAN-I-ENV, current command environment: Clusterwide on local cluster Username SYSTEM will be used on nonlocal nodes SYSMAN> SET PROFILE/PRIVILEGES=(OPER,SYSPRV) SYSMAN> DO MCR NCP SET EXECUTOR STATE OFF %SYSMAN-I-OUTPUT, command execution on node X... . . . SYSMAN> DO MCR NCP DEFINE EXECUTOR ALIAS INCOMING ENABLED %SYSMAN-I-OUTPUT, command execution on node X... . . . SYSMAN> DO @SYS$MANAGER:STARTNET.COM %SYSMAN-I-OUTPUT, command execution on node X... . . .
Note: VSI does not recommend enabling alias operations for satellite nodes.
Reference: For more details about DECnet for OpenVMS networking and cluster alias, see the VSI OpenVMS DECnet Networking Manual and VSI OpenVMS DECnet Network Management Utilities. For equivalent information about DECnet–Plus, see the DECnet–Plus documentation.
4.5.10. Configuring TCP/IP
For information on how to configure and start TCP/IP, see the VSI TCP/IP Services for OpenVMS Installation and Configuration and the HP TCP/IP Services for OpenVMS Version 5.7 Release Notes.
Chapter 5. Preparing a Shared Environment
In any OpenVMS Cluster environment, it is best to share resources as much as possible. Resource sharing facilitates workload balancing because work can be distributed across the cluster.
5.1. Shareable Resources
Shareable Resources | Description |
---|---|
System disks |
All members of the same architecture?can share a single system disk, each member can have its own system disk, or members can use a combination of both methods. |
Data disks |
All members can share any data disks. For local disks, access is limited to the local node unless you explicitly setup the disks to be cluster accessible by means of the MSCP server. |
Tape drives |
All members can share tape drives. (Note that this does not imply that all members can have simultaneous access.)For local tape drives, access is limited to the local node unless you explicitly set up the tapes to be cluster accessible by means of the TMSCP server. Only DSA tapes can be served to all OpenVMS Cluster members. |
Batch and print queues |
Users can submit batch jobs to any queue in the OpenVMS Cluster, regardless of the processor on which the job will actually execute. Generic queues can balance the load among the available processors. |
Applications |
Most applications work in an OpenVMS Cluster just as they do on a single system. Application designers can also create applications that run simultaneously on multiple OpenVMS Cluster nodes, which share data in a file. |
User authorization files |
All nodes can use either a common user authorization file (UAF)for the same access on all systems or multiple UAFs to enable node-specific quotas. If a common UAF is used, all user passwords, directories, limits, quotas, and privileges are the same on all systems. |
5.1.1. Local Resources
Nonshareable Resources | Description |
---|---|
Memory |
Each OpenVMS Cluster member maintains its own memory. |
User processes |
When a user process is created on an OpenVMS Cluster member, the process must complete on that computer, using local memory. |
Printers |
A printer that does not accept input through queues is used only by the OpenVMS Cluster member to which it is attached. A printer that accepts input through queues is accessible by any OpenVMS Cluster member. |
5.1.2. Sample Configuration
Figure 5.1, “Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)” shows an OpenVMS Cluster system that shares FC SAN storage between the Integrity servers and Alpha systems. Each architecture has its own system disk.

5.1.3. Storage in a Mixed-Architecture Cluster
This section describes the rules pertaining to storage, including system disks, in a mixed-architecture cluster consisting of OpenVMS Integrity servers and OpenVMS Alpha systems.
Figure 5.2, “Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)” is a simplified version of a mixed-architecture cluster of OpenVMS Integrity servers and OpenVMS Alpha systems with locally attached storage and a shared Storage Area Network (SAN).

Must have an Integrity server system disk, either a local disk or a shared Fibre Channel disk.
Can use served Alpha disks and served Alpha tapes.
Can use SAN disks and tapes.
Can share the same SAN data disk with Alpha systems.
Can serve disks and tapes to other cluster members, both Integrity servers and Alpha systems.
Must have an Alpha system disk, which can be shared with other clustered Alpha systems.
Can use locally attached tapes and disks.
Can serve disks and tapes to both Integrity servers and Alpha systems.
Can use Integrity servers served data disks.
Can use SAN disks and tapes.
Can share the same SAN data disk with Integrity server systems.
5.2. Common-Environment and Multiple-Environment Clusters
Depending on your processing needs, you can prepare either an environment in which all environmental files are shared clusterwide or an environment in which some files are shared clusterwide while others are accessible only by certain computers.
Cluster Type | Characteristics | Advantages |
---|---|---|
Common environment | ||
Operating environment is identical on all nodes in the OpenVMS Cluster. |
The environment is set up so that:
|
Easier to manage because you use a common version of each system file. |
Multiple environment | ||
Operating environment can vary from node to node. |
An individual processor or a subset of processors are set up
to:
|
Effective when you want to share some data among computers but you also want certain computers to serve specialized needs. |
5.3. Directory Structure on Common System Disks
The installation or upgrade procedure for your operating system generates a common system disk, on which most operating system and optional product files are stored in a system root directory.
5.3.1. Directory Roots
The system disk directory structure is the same on Integrity servers and Alpha systems. Whether the system disk is for an Integrity server system or Alpha, the entire directory structure—that is, the common root plus each computer's local root is stored on the same disk. After the installation or upgrade completes, you use the CLUSTER_CONFIG.COM or CLUSTER_CONFIG_LAN.COM command procedure described in Chapter 8, Configuring an OpenVMS Cluster System to create a local root for each new computer to use when booting into the cluster.
In addition to the usual system directories, each local root contains a [SYS n.SYSCOMMON] directory that is a directory alias for [VMS$COMMON], the cluster common root directory in which cluster common files actually reside. When you add a computer to the cluster, the com procedure defines the common root directory alias.
5.3.2. Directory Structure Example
Figure 5.3, “Directory Structure on a Common System Disk” illustrates the directory structure set up for computers JUPITR and SATURN, which are run from a common system disk. The disk's master file directory (MFD) contains the local roots (SYS0 for JUPITR, SYS1 for SATURN) and the cluster common root directory, [VMS$COMMON].

5.3.3. Search Order
The logical name SYS$SYSROOT is defined as a search list that points first to a local root (SYS$SYSDEVICE:[SYS0.SYSEXE]) and then to the common root (SYS$COMMON:[SYSEXE]). Thus, the logical names for the system directories (SYS$SYSTEM, SYS$LIBRARY, SYS$MANAGER, and so forth) point to two directories.
Figure 5.4, “File Search Order on Common System Disk” shows how directories on a common system disk are searched when the logical name SYS$SYSTEM is used in file specifications.

Important: Keep this search order in mind when you manipulate system files on a common system disk. Computer-specific files must always reside and be updated in the appropriate computer's system subdirectory.
- MODPARAMS.DAT must reside in SYS$SPECIFIC:[SYSEXE], which is[SYS0.SYSEXE] on JUPITR, and in [SYS1.SYSEXE] on SATURN. Thus, to create a new MODPARAMS.DAT file for JUPITR when logged in on JUPITR, enter the following command:
$ EDIT SYS$SPECIFIC:[SYSEXE]MODPARAMS.DAT
Once the file is created, you can use the following command to modify it when logged on to JUPITR:$ EDIT SYS$SYSTEM:MODPARAMS.DAT
Note that if a MODPARAMS.DAT file does not exist in JUPITR's SYS$SPECIFIC:[SYSEXE] directory when you enter this command, but there is a MODPARAMS.DAT file in the directory SYS$COMMON:[SYSEXE], the command edits the MODPARAMS.DAT file in the common directory. If there is no MODPARAMS.DAT file in either directory, the command creates the file in JUPITR's SYS$SPECIFIC:[SYSEXE] directory.
- To modify JUPITR's MODPARAMS.DAT when logged in on any other computer that boots from the same common system disk, enter the following command:
$ EDIT SYS$SYSDEVICE:[SYS0.SYSEXE]MODPARAMS.DAT
- To modify records in the cluster common system authorization file in a cluster with a single, cluster-common system disk, enter the following commands on any computer:
$ SET DEFAULT SYS$COMMON:[SYSEXE]$ RUN SYS$SYSTEM:AUTHORIZE
- To modify records in a computer-specific system authorization file when logged in to another computer that boots from the same cluster common system disk, you must set your default directory to the specific computer. For example, if you have set up a computer-specific system authorization file(SYSUAF.DAT) for computer JUPITR, you must set your default directory to JUPITR's computer-specific [SYSEXE] directory before invoking AUTHORIZE, as follows:
$ SET DEFAULT SYS$SYSDEVICE:[SYS0.SYSEXE] $ RUN SYS$SYSTEM:AUTHORIZE
5.4. Clusterwide Logical Names
Clusterwide logical names, introduced in OpenVMS Version 7.2, extend the convenience and ease-of-use features of shareable logical names to OpenVMS Cluster systems. Clusterwide logical names are available on OpenVMS Integrity servers and OpenVMS Alpha systems, in a single or a mixed architecture OpenVMS Cluster.
Existing applications can take advantage of clusterwide logical names without any changes to the application code. Only a minor modification to the logical name tables referenced by the application (directly or indirectly) is required.
New logical names are local by default. Clusterwide is an attribute of a logical name table. In order for a new logical name to be clusterwide, it must be created in a clusterwide logical name table.
When a new node joins the cluster, it automatically receives the current set of clusterwide logical names.
When a clusterwide logical name or name table is created, modified, or deleted, the change is automatically propagated to every other node in the cluster running OpenVMS Version 7.2 or later. Modifications include security profile changes to a clusterwide table.
Translations are done locally so there is minimal performance degradation for clusterwide name translations.
Because LNM$CLUSTER_TABLE and LNM$SYSCLUSTER_TABLE exist on all systems running OpenVMS Version 7.2 or later, the programs and command procedures that use clusterwide logical names can be developed, tested, and run on nonclustered systems.
5.4.1. Default Clusterwide Logical Name Tables
Name |
Purpose |
---|---|
LNM$SYSCLUSTER_TABLE |
The default table for clusterwide system logical names. It is empty when shipped. This table is provided for system managers who want to use clusterwide logical names to customize their environments. The names in this table are available to anyone translating a logical name using SHOW LOGICAL/SYSTEM, specifying a table name of LNM$SYSTEM, or LNM$DCL_LOGICAL (DCL's default table search list), or LNM$FILE_DEV (system and RMS default). |
LNM$SYSCLUSTER |
The logical name for LNM$SYSCLUSTER_TABLE. It is provided for convenience in referencing LNM$SYSCLUSTER_TABLE. It is consistent in format with LNM$SYSTEM_TABLE and its logical name, LNM$SYSTEM. |
LNM$CLUSTER_TABLE |
The parent table for all clusterwide logical name tables, including LNM$SYSCLUSTER_TABLE. When you create a new table using LNM$CLUSTER_TABLE as the parent table, the new table will be available clusterwide. |
LNM$CLUSTER |
The logical name for LNM$CLUSTER_TABLE. It is provided for convenience in referencing LNM$CLUSTER_TABLE. |
5.4.2. Translation Order
The definition of LNM$SYSTEM has been expanded to include LNM$SYSCLUSTER. When a system logical name is translated, the search order is LNM$SYSTEM_TABLE, LNM$SYSCLUSTER_TABLE. Because the definitions for the system default table names, LNM$FILE_DEV and LNM$DCL_LOGICALS, include LNM$SYSTEM, translations using those default tables include definitions in LNM$SYSCLUSTER.
The current precedence order for resolving logical names is preserved. Clusterwide logical names that are translated against LNM$FILE_DEV are resolved last, after system logical names. The precedence order, from first to last, is process → job → group → system → cluster, as shown in Figure 5.5, “Translation Order Specified by LNM$FILE_DEV”.

5.4.3. Creating Clusterwide Logical Name Tables
For a multiprocess clusterwide application to use
For members of a UIC group to share
To create a clusterwide logical name table, you must have create (C) access to the parent table and write (W) access to LNM$SYSTEM_DIRECTORY, or the SYSPRV(system) privilege.
A shareable logical name table has UIC-based protection. Each class of user(system (S), owner (O), group (G), and world (W)) can be granted four types of access: read (R), write (W), create (C), or delete (D).
You can create additional clusterwide logical name tables in the same way that you can create additional process, job, and group logical name tables – with the CREATE/NAME_TABLE command or with the $CRELNT system service. When creating a clusterwide logical name table, you must specify the /PARENT_TABLE qualifier and provide a value for the qualifier that is a clusterwide table name. Any existing clusterwide table used as the parent table will make the new table clusterwide.
$ CREATE/NAME_TABLE/PARENT_TABLE=LNM$CLUSTER_TABLE -
_$ new-clusterwide-logical-name-table
5.4.4. Alias Collisions Involving Clusterwide Logical Name Tables
Collision Type |
Outcome |
---|---|
Creating a local table with same name and access mode as an existing clusterwide table |
New local table is not created. The condition value
|
Creating a clusterwide table with same name and access mode as an existing local table |
New clusterwide table is created. The condition value
SS$_LNMCREATED is returned, which
means that the logical name table was created. The local
table and its names are deleted. If the clusterwide table
was created with the DCL command DEFINE, a message is
displayed:DCL-I-TABSUPER, previous table table_name
has been superseded If the clusterwide table was created with the $CRELNT
system service,$CRELNT returns the condition value:
|
Creating a clusterwide table with same name and access mode as an existing clusterwide table |
New clusterwide table is not created. The condition value
|
5.4.5. Creating Clusterwide Logical Names
To create a clusterwide logical name, you must have write (W) access to the table in which the logical name is to be entered, or SYSNAM privilege if you are creating clusterwide logical names only in LNM$SYSCLUSTER. Unless you specify an access mode (user, supervisor, and so on), the access mode of the logical name you create defaults to the access mode from which the name was created. If you created the name with a DCL command, the access mode defaults to supervisor mode. If you created the name with a program, the access mode typically defaults to user mode.
When you create a clusterwide logical name, you must include the name of a clusterwide logical name table in the definition of the logical name. You can create clusterwide logical names by using DCL commands or with the $CRELNM system service.
$ DEFINE/TABLE=LNM$CLUSTER_TABLE logical-name equivalence-string
$ DEFINE/TABLE=new-clusterwide-logical-name-table logical-name - _$ equivalence-string
Note
If you attempt to create a new clusterwide logical name with the same access mode and identical equivalence names and attributes as an existing clusterwide logical name, the existing name is not deleted, and no messages are sent to remote nodes. This behavior differs from similar attempts for other types of logical names, which delete the existing name and create the new one. For clusterwide logical names, this difference is a performance enhancement.
The condition value SS$_NORMAL
is returned. The service
completed successfully, but the new logical name was not created.
5.4.6. Management Guidelines
Do not use certain logical names clusterwide.
he following logical names are not valid for clusterwide use:Mailbox names, because mailbox devices are local to a node.
SYS$NODE and SYS$NODE_FULLNAME must be in LNM$SYSTEM_TABLE and are node specific.
LMF$LICENSE_TABLE.
Do not redefine LNM$SYSTEM.
LNM$SYSTEM is now defined as LNM$SYSTEM_TABLE, LNM$SYSCLUSTER_TABLE. Do not reverse the order of these two tables. If you do, then any names created using the /SYSTEM qualifier or in LNM$SYSTEM would go in LNM$SYSCLUSTER_TABLE and be clusterwide. Various system failures would result. For example, the MOUNT/SYSTEM command would attempt to create a clusterwide logical name for a mounted volume, which would result in an error.
Keep LNM$SYSTEM contents in LNM$SYSTEM.
Do not merge the logical names in LNM$SYSTEM into LNM$SYSCLUSTER. Many system logical names in LNM$SYSTEM contain system roots and either node-specific devices, or node-specific directories, or both.
Adopt naming conventions for logical names used at your site.
To avoid confusion and name conflicts, develop one naming convention for system-specific logical names and another for clusterwide logical names.
Avoid using the dollar sign ($) in your own site's logical names, because OpenVMS software uses it in its names.
Be aware that clusterwide logical name operations will stall when the clusterwide logical name database is not consistent.
This can occur during system initialization when the system's clusterwide logical name database is not completely initialized. It can also occur when the cluster server process has not finished updating the clusterwide logical name database, or during resynchronization after nodes enter or leave the cluster. As soon as consistency is reestablished, the processing of clusterwide logical name operations resumes.
5.4.7. Using Clusterwide Logical Names in Applications
The $TRNLNM system service and the $GETSYI system service provide attributes that are specific to clusterwide logical names. This section describes those attributes. It also describes the use of$CRELNT as it pertains to creating a clusterwide table. For more information about using logical names in applications, refer to the VSI OpenVMS Programming Concepts Manual.
5.4.7.1. Clusterwide Attributes for $TRNLNM System Service
LNM$V_CLUSTERWIDE
LNM$M_INTERLOCKED
LNM$V_CLUSTERWIDE is an output attribute to be returned in the item list if you asked for the LNM$_ATTRIBUTES item for a logical name that is clusterwide.
LNM$M_INTERLOCKED is an attr argument bit that can be set to ensure that any clusterwide logical name modifications in progress are completed before the name is translated. LNM$M_INTERLOCKED is not set by default. If your application requires translation using the most recent definition of a clusterwide logical name, use this attribute to ensure that the translation is stalled until all pending modifications have been made.
On a single system, when one process modifies the shareable part of the logical name database, the change is visible immediately to other processes on that node. Moreover, while the modification is in progress, no other process can translate or modify shareable logical names.
In contrast, when one process modifies the clusterwide logical name database, the change is visible immediately on that node, but it takes a short time for the change to be propagated to other nodes. By default, translations of clusterwide logical names are not stalled. Therefore, it is possible for processes on different nodes to translate a logical name and get different equivalence names when modifications are in progress.
The use of LNM$M_INTERLOCKED guarantees that your application will receive the most recent definition of a clusterwide logical name.
5.4.7.2. Clusterwide Attribute for $GETSYI System Service
The clusterwide attribute, SYI$_CWLOGICALS, has been added to the $GETSYI system service. When you specify SYI$_CWLOGICALS, $GETSYI returns the value 1 if the clusterwide logical name database has been initialized on the CPU, or the value 0 if it has not been initialized. Because this number is a Boolean value (1 or 0), the buffer length field in the item descriptor should specify 1 (byte). On a nonclustered system, the value of SYI$_CWLOGICALS is always 0.
5.4.7.3. Creating Clusterwide Tables with the $CRELNT System Service
When creating a clusterwide table, the $CRELNT requester must supply a table name. OpenVMS does not supply a default name for clusterwide tables because the use of default names enables a process without the SYSPRV privilege to create a shareable table.
5.4.8. Defining and Accessing Clusterwide Logical Names
Initializing the clusterwide logical name database on a booting node requires sending a message to another node and having its CLUSTER_SERVER process reply with one or messages containing a description of the database. The CLUSTER_SERVER process on the booting node requests system services to create the equivalent names and tables. How long this initialization takes varies with conditions such as the size of the clusterwide logical name database, the speed of the cluster interconnect, and the responsiveness of the CLUSTER_SERVER process on the responding node.
Until a booting node's copy of the clusterwide logical name database is consistent with the logical name databases of the rest of the cluster, any attempt on the booting node to create or delete clusterwide names or tables is stalled transparently. Because translations are not stalled by default, any attempt to translate a clusterwide name before the database is consistent may fail or succeed, depending on timing. To stall a translation until the database is consistent, specify the F$TRNLNM CASE argument as INTERLOCKED.
5.4.8.1. Defining Clusterwide Logical Names in SYSTARTUP_VMS.COM
In general, system managers edit the SYLOGICALS.COM command procedure to define site-specific logical names that take effect at system startup. However, VSI recommends that, if possible, clusterwide logical names be defined in the SYSTARTUP_VMS.COM command procedure instead with the exception of those logical names discussed in Section 5.4.8.2, “Defining Certain Logical Names in SYLOGICALS.COM”. The reason for defining clusterwide logical names in SYSTARTUP_VMS.COM is that SYSTARTUP_VMS.COM is run at a much later stage in the booting process than SYLOGICALS.COM.
OpenVMS startup is single streamed and synchronous except for actions taken by created processes, such as the CLUSTER_SERVER process. Although the CLUSTER_SERVER process is created very early in startup, it is possible that when SYLOGICALS.COM is executed, the booting node's copy of the clusterwide logical name database has not been fully initialized. In such a case, a clusterwide definition in SYLOGICALS.COM would stall startup and increase the time it takes for the system to become operational.
OpenVMS will ensure that the clusterwide database has been initialized before SYSTARTUP_VMS.COM is executed.
5.4.8.2. Defining Certain Logical Names in SYLOGICALS.COM
To be effective, certain logical names, such as LMF$LICENSE, NET$PROXY, and VMS$OBJECTS must be defined earlier in startup than when SYSTARTUP_VMS.COM is invoked. Most such names are defined in SYLOGICALS.COM, with the exception of VMS$OBJECTS, which is defined in SYSECURITY.COM, and any names defined in SYCONFIG.COM.
Although VSI recommends defining clusterwide logical names in SYSTARTUP_VMS.COM, to define these names to be clusterwide, you must do so in SYLOGICALS.COM or SYSECURITY.COM. Note that doing this may increase startup time.
Alternatively, you can take the traditional approach and define these names as systemwide logical names with the same definition on every node.
5.4.8.3. Using Conditional Definitions for Startup Command Procedures
$ IF F$TRNLNM("CLUSTER_APPS") .EQS. "" THEN - _$ DEFINE/TABLE=LNM$SYSCLUSTER/EXEC CLUSTER_APPS - _$ $1$DKA500:[COMMON_APPS]
A conditional definition can prevent unpleasant surprises. For example, suppose a system manager redefines a name that is also defined in SYSTARTUP_VMS.COM but does not edit SYSTARTUP_VMS.COM because the new definition is temporary. If a new node joins the cluster, the new node would initially receive the new definition. However, when the new node executes SYSTARTUP_VMS.COM, it will cause all the nodes in the cluster, including itself, to revert to the original value.
$ IF F$TRNLNM("CLUSTER_APPS",,,,"INTERLOCKED") .EQS. "" THEN - _$ DEFINE/TABLE=LNM$SYSCLUSTER/EXEC CLUSTER_APPS - _$ $1$DKA500:[COMMON_APPS]
Note
F$GETSYI ("CWLOGICALS") always returns a value of FALSE on a noncluster system. Procedures that are designed to run in both clustered and nonclustered environments should first determine whether they are in a cluster and, if so, then determine whether clusterwide logical names are initialized.
5.4.9. Displaying Clusterwide Logical Names
$ SHOW LOGICAL/CLUSTER (LNM$CLUSTER_TABLE) (LNM$SYSCLUSTER_TABLE) "MSCPMOUNT$_AMALFI_LAST" = "2005-10-10 14:25:03.74" "MSCPMOUNT$_AMALFI_LOGINTIM" = " 8-OCT-2005 01:02:22.17" "MSCPMOUNT$_AMALFI_NEXT" = "2005-10-10 14:40:03.74" "MSCPMOUNT$_AMALFI_PID" = "26200462" . . . "MSCPMOUNT$_ETNA_LAST" = "2005-10-10 14:25:18.78" "MSCPMOUNT$_ETNA_LOGINTIM" = " 8-OCT-2005 07:44:37.89" "MSCPMOUNT$_ETNA_NEXT" = "2005-10-10 14:40:18.79" "MSCPMOUNT$_ETNA_PID" = "26A0044E" . . . "MSCPMOUNT$_MILAN_LAST" = "2005-10-10 14:25:19.64" "MSCPMOUNT$_MILAN_LOGINTIM" = " 8-OCT-2005 07:22:08.05" "MSCPMOUNT$_MILAN_NEXT" = "2005-10-10 14:40:19.64" "MSCPMOUNT$_MILAN_PID" = "26600458" . . . "MSCPMOUNT$_ORVIET_LAST" = "2005-10-10 14:29:25.94" "MSCPMOUNT$_ORVIET_LOGINTIM" = "30-SEP-2005 09:38:27.38" "MSCPMOUNT$_ORVIET_NEXT" = "2005-10-10 14:44:26.61" "MSCPMOUNT$_ORVIET_PID" = "25600139" . . . "MSCPMOUNT$_TURIN_LAST" = "2005-10-10 14:39:59.59" "MSCPMOUNT$_TURIN_LOGINTIM" = "10-OCT-2005 09:22:48.46" "MSCPMOUNT$_TURIN_NEXT" = "2005-10-10 14:54:59.59" "MSCPMOUNT$_TURIN_PID" = "2760012C" "PREPOPULATE_NEXT_STREAM$IGNORE_BUILD_MASTER_944" = "1" (CLU$ICC_ORBS_AMALFI) "ICC$ORB_ICC$PID_26200450_U" = "T" = "M\.v....k...............æ...æ...þ...þ.....AMALFI::ICC$PID_26200450_U....." "ICC$ORB_REG$SERVER_E" = "T" = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_E044........" "ICC$ORB_REG$SERVER_K" = "T" = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_K044........" "ICC$ORB_REG$SERVER_U" = "T" = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_U044........" (CLU$ICC_ORBS_ETNA) (CLU$ICC_ORBS_MILAN) (CLU$ICC_ORBS_ORVIET) "ICC$ORB_ICC$PID_26000450_U" = "T" = "VQ.p....k...............æ...æ...þ...þ.....ETNA::ICC$PID_26000450_U......." (CLU$ICC_ORBS_TURIN) . . . (ICC$REGISTRY_TABLE)
5.5. Coordinating Startup Command Procedures
Immediately after a computer boots, it runs the site-independent command procedure SYS$SYSTEM:STARTUP.COM to start up the system and control the sequence of startup events. The STARTUP.COM procedure calls a number of other startup command procedures that perform cluster-specific and node-specific tasks.
The following sections describe how, by setting up appropriate cluster-specific startup command procedures and other system files, you can prepare the OpenVMS Cluster operating environment on the first installed computer before adding other computers to the cluster.
Reference: See also the VSI OpenVMS System Manager's Manual for more information about startup command procedures.
5.5.1. OpenVMS Startup Procedures
Procedure Name | Invoked by | Function |
---|---|---|
|
|
A file to which you add commands to install page and swap files (other than the primary page and swap files that are installed automatically). |
|
|
Connects special devices and loads device I/O drivers. |
|
|
Defines the location of the security audit and archive files before it starts the security audit server. |
|
|
Creates systemwide logical names, and defines system components as executive-mode logical names. (Clusterwide logical names should be defined in SYSTARTUP_VMS.COM.) Cluster common disks can be mounted at the end of this procedure. |
|
|
Performs many of the following startup and login
functions:
|
The directory SYS$COMMON:[SYSMGR] contains a template file for each command procedure that you can edit. Use the command procedure templates (in SYS$COMMON:[SYSMGR]*.TEMPLATE) as examples for customization of your system's startup and login characteristics.
5.5.2. Building Startup Procedures
The first step in preparing an OpenVMS Cluster shared environment is to build a SYSTARTUP_VMS command procedure. Each computer executes the procedure at startup time to define the operating environment.
Step | Action |
---|---|
1 |
In each computer's SYS$SPECIFIC:[SYSMGR] directory, edit
the SYSTARTUP_VMS.TEMPLATE file to set up a
SYSTARTUP_VMS.COM procedure that:
|
2 |
Build a common command procedure that includes startup
commands that you want to be common to all computers. The
common procedure might contain commands that:
Note: You might choose to build these commands into individual command procedures that are invoked from the common procedure. For example, the MSCPMOUNT.COM file in the SYS$EXAMPLES directory is a sample common command procedure that contains commands typically used to mount cluster disks. The example includes comments explaining each phase of the procedure. |
3 |
Place the common procedure in the SYS$COMMON:[SYSMGR] directory on a common system disk or other cluster-accessible disk. Important: The common procedure is usually located in the SYS$COMMON:[SYSMGR] directory on a common system disk but can reside on any disk, provided that the disk is cluster accessible and is mounted when the procedure is invoked. If you create a copy of the common procedure for each computer, you must remember to update each copy whenever you make changes. |
5.5.3. Combining Existing Procedures
To build startup procedures for an OpenVMS Cluster system in which existing computers are to be combined, you should compare both the computer-specific SYSTARTUP_VMS and the common startup command procedures on each computer and make any adjustments required. For example, you can compare the procedures from each computer and include commands that define the same logical names in your common SYSTARTUP_VMS command procedure.
After you have chosen which commands to make common, you can build the common procedures on one of the OpenVMS Cluster computers.
5.5.4. Using Multiple Startup Procedures
To define a multiple-environment cluster, you set up computer-specific versions of one or more system files. For example, if you want to give users larger working set quotas on URANUS, you would create a computer-specific version of SYSUAF.DAT and place that file in system's root directory. That directory can be located in URANUS's root on a common system disk or on an individual system disk that you have set up on URANUS.
Step |
Action |
---|---|
1 |
Include in SYSTARTUP_VMS.COM elements that you want to remain unique to a computer, such as commands to define computer-specific logical names and symbols. |
2 |
Place these files in the SYS$SPECIFIC root on each computer. |
Example: Consider a three-member cluster consisting of computers JUPITR, SATURN, and PLUTO. The time sharing environments on JUPITR and SATURN are the same. However, PLUTO runs applications for a specific user group. In this cluster, you would create a common SYSTARTUP_VMS command procedure for JUPITR and SATURN that defines identical environments on these computers. But the command procedure for PLUTO would be different; it would include commands to define PLUTO's special application environment.
5.6. Providing OpenVMS Cluster System Security
The OpenVMS security subsystem ensures that all authorization information and object security profiles are consistent across all nodes in the cluster. The OpenVMS operating system does not support multiple security domains because the operating system cannot enforce a level of separation needed to support different security domains on separate cluster members.
5.6.1. Security Checks
Authorized users can have processes executing on any OpenVMS Cluster member.
A process, acting on behalf of an authorized individual, requests access to a cluster object.
A coordinating node determines the outcome by comparing its copy of the common authorization database with the security profile for the object being accessed.
The OpenVMS operating system provides the same strategy for the protection of files and queues, and further incorporates all other cluster-visible objects, such as devices, volumes, and lock resource domains.
Starting with OpenVMS Version 7.3, the operating system provides clusterwide intrusion detection, which extends protection against attacks of all types throughout the cluster. The intrusion data and information from each system is integrated to protect the cluster as a whole. Prior to Version 7.3, each system was protected individually.
The SECURITY_POLICY system parameter controls whether a local or a clusterwide intrusion database is maintained for each system. The default setting is for a clusterwide database, which contains all unauthorized attempts and the state of any intrusion events for all cluster members that are using this setting. Cluster members using the clusterwide intrusion database are made aware if a cluster member is under attack or has any intrusion events recorded. Events recorded on one system can cause another system in the cluster to take restrictive action. (For example, the person attempting to log in is monitored more closely and limited to a certain number of login retries within a limited period of time. Once a person exceeds either the retry or time limitation, he or she cannot log in).
Actions of the cluster manager in setting up an OpenVMS Cluster system can affect the security operations of the system. You can facilitate OpenVMS Cluster security management using the suggestions discussed in the following sections.
The easiest way to ensure a single security domain is to maintain a single copy of each of the following files on one or more disks that are accessible from anywhere in the OpenVMS Cluster system. When a cluster is configured with multiple system disks, you can use system logical names (as shown in Section 5.8, “Coordinating System Files”) to ensure that only a single copy of each file exists.
- SYS$MANAGER:VMS$AUDIT_SERVER.DAT
- SYS$SYSTEM:NETOBJECT.DAT
- SYS$SYSTEM:NETPROXY.DAT
- TCPIP$PROXY.DAT
- SYS$SYSTEM:PE$IP_CONFIG.DAT
- SYS$SYSTEM:QMAN$MASTER.DAT
- SYS$SYSTEM:RIGHTSLIST.DAT
- SYS$SYSTEM:SYSALF.DAT
- SYS$SYSTEM:SYSUAF.DAT
- SYS$SYSTEM:SYSUAFALT.DAT
- SYS$SYSTEM:VMS$PASSWORD_HISTORY.DATA
- SYS$SYSTEM:VMSMAIL_PROFILE.DATA
- SYS$LIBRARY:VMS$PASSWORD_DICTIONARY.DATA
- SYS$LIBRARY:VMS$PASSWORD_POLICY.EXE
Note: Using shared files is not the only way of achieving a single security domain. You may need to use multiple copies of one or more of these files on different nodes in a cluster. For example, on Alpha nodes you may choose to deploy system-specific user authorization files (SYSUAFs) to allow for different memory management working-set quotas among different nodes. Such configurations are fully supported as long as the security information available to each node in the cluster is identical.
5.6.2. Files Relevant to OpenVMS Cluster Security
Table 5.3, “Security Files” describes the security-relevant portions of the files that must be common across all cluster members to ensure that a single security domain exists.
Some of these files are created only on request and may not exist in all configurations.
A file can be absent on one node only if it is absent on all nodes.
As soon as a required file is created on one node, it must be created or commonly referenced on all remaining cluster nodes.
Table Keyword |
Meaning |
---|---|
Required |
The file contains some data that must be kept common across all cluster members to ensure that a single security environment exists. |
Recommended |
The file contains data that should be kept common at the discretion of the site security administrator or system manager. Nonetheless, VSI recommends that you synchronize the recommended files. |
File Name | Contains | |
---|---|---|
CLUSTER_AUTHORIZE.DAT |
The cluster authorization file, SYS$COMMON:[SYSEXE]CLUSTER_AUTHORIZE.DAT, contains the cluster group number in a disorderly form and the cluster password. The CLUSTER_AUTHORIZE.DAT file is accessible only to users with the SYSPRV privilege. | |
PE$IP_CONFIG.DAT [recommended] |
For cluster over IP configurations, which are using IP unicast, the remote node IP address should be present in the existing cluster members file in the SYS$SYSTEM:PE$IP_CONFIG.DAT file. Remote nodes in a different IP multicast domain can use the IP unicast messaging technique to join the Cluster. | |
VMS$AUDIT_SERVER.DAT [recommended] |
Information related to security auditing. Among the information contained is the list of enabled security auditing events and the destination of the system security audit journal file. When more than one copy of this file exists, all copies should be update dafter any SET AUDIT command. OpenVMS Cluster system managers should ensure that the
name assigned to the security audit journal file resolves to
the following location:
SYS$COMMON:[SYSMGR]SECURITY.AUDIT$JOURNAL Rule: If you need to relocate the audit journal file somewhere other than the system disk (or if you have multiple system disks), you should redirect the audit journal uniformly across all nodes in the cluster. Use the command SETAUDIT/JOURNAL=SECURITY/DESTINATION= file-name, specifying a file name that resolves to the same file throughout the cluster. Changes are automatically made in the audit server database, SYS$MANAGER:VMS$AUDIT_SERVER.DAT. This database also identifies which events are enabled and how to monitor the audit system's use of resources, and restores audit system settings each time the system is rebooted. Caution: Failure to synchronize multiple copies of this file properly may result in partitioned auditing domains. Reference: For more information, see the VSI OpenVMS Guide to System Security. | |
NETOBJECT.DAT [required] |
The DECnet object database. Among the information contained in this file is the list of known DECnet server accounts and passwords. When more than one copy of this file exists, all copies must be updated after every use of the NCP commands SET OBJECT or DEFINE OBJECT. Caution: Failure to synchronize multiple copies of this file properly may result in unexplained network login failures and unauthorized network access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. Reference: Refer to the DECnet–Plus documentation for equivalent NCL command information. | |
NETPROXY.DAT and NET$PROXY.DAT [required] |
The network proxy database. It is maintained by the OpenVMS Authorize utility. When more than one copy of this file exists, all copies must be updated after any UAF proxy command. Note: The NET$PROXY.DAT and NETPROXY.DAT files are equivalent;NET$PROXY.DAT is for DECnet–Plus implementations and NETPROXY.DAT is for DECnet for OpenVMS implementations. Caution: Failure to synchronize multiple copies of this file properly may result in unexplained network login failures and unauthorized network access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. Reference: Appendix B, Building Common Files discusses how to consolidate several NETPROXY.DAT and RIGHTSLIST.DAT files. | |
TCPIP$PROXY.DAT |
This database provides OpenVMS identities for remote NFS clients and UNIX-style identifiers for local NFS client users;provides proxy accounts for remote processes. For more information about this file, see the VSI TCP/IP Services for OpenVMS Management. | |
QMAN$MASTER.DAT [required] |
The master queue manager database. This file contains the security information for all shared batch and print queues. Rule: If two or more nodes are to participate in a shared queuing system, a single copy of this file must be maintained on a shared disk. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. | |
RIGHTSLIST.DAT [required] |
The rights identifier database. It is maintained by the OpenVMS Authorize utility and by various rights identifier system services. When more than one copy of this file exists, all copies must be updated after any change to any identifier or holder records. Caution: Failure to synchronize multiple copies of this file properly may result in unauthorized system access and unauthorized access to protected objects. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. Reference: Appendix B, Building Common Files discusses how to consolidate several NETPROXY.DAT and RIGHTSLIST.DAT files. | |
SYSALF.DAT [required] |
The system Auto login facility database. It is maintained by the OpenVMS SYSMAN utility. When more than one copy of this file exists, all copies must be updated after any SYSMAN ALF command. Note: This file may not exist in all configurations. Caution: Failure to synchronize multiple copies of this file properly may result in unexplained login failures and unauthorized system access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. | |
SYSUAF.DAT [required] |
The system user authorization file. It is maintained by the OpenVMS Authorize utility and is modifiable via the$SETUAI system service. When more than one copy of this file exists, you must ensure that the SYSUAF and associated $SETUAI item codes are synchronized for each user record. The following table shows the fields in SYSUAF and their associated $SETUAI item codes. | |
Internal Field Name |
$SETUAI Item Code | |
UAF$R_DEF_CLASS |
UAI$_DEF_CLASS | |
UAF$Q_DEF_PRIV |
UAI$_DEF_PRIV | |
UAF$B_DIALUP_ACCESS_P |
UAI$_LOCAL_ACCESS_P | |
UAF$B_LOCAL_ACCESS_S |
UAI$_DIALUP_ACCESS_S | |
UAF$B_ENCRYPT |
UAI$_ENCRYPT | |
UAF$B_ENCRYPT2 |
UAI$_ENCRYPT2 | |
UAF$Q_EXPIRATION |
UAI$_EXPIRATION | |
UAF$L_FLAGS |
UAI$_FLAGS | |
UAF$B_LOCAL_ACCESS_P |
UAI$_LOCAL_ACCESS_P | |
UAF$B_LOCAL_ACCESS_S |
UAI$_LOCAL_ACCESS_S | |
UAF$B_NETWORK_ACCESS_P |
UAI$_NETWORK_ACCESS_P | |
UAF$B_NETWORK_ACCESS_S |
UAI$_NETWORK_ACCESS_S | |
UAF$B_PRIME_DAYS |
UAI$_PRIMEDAYS | |
UAF$Q_PRIV |
UAI$_PRIV | |
UAF$Q_PWD |
UAI$_PWD | |
UAF$Q_PWD2 |
UAI$_PWD2 | |
UAF$Q_PWD_DATE |
UAI$_PWD_DATE | |
UAF$Q_PWD2_DATE |
UAI$_PWD2_DATE | |
UAF$B_PWD_LENGTH |
UAI$_PWD_LENGTH | |
UAF$Q_PWD_LIFETIME |
UAI$_PWD_LIFETIME | |
UAF$B_REMOTE_ACCESS_P |
UAI$_REMOTE_ACCESS_P | |
UAF$B_REMOTE_ACCESS_S |
UAI$_REMOTE_ACCESS_S | |
UAF$R_MAX_CLASS |
UAI$_MAX_CLASS | |
UAF$R_MIN_CLASS |
UAI$_MIN_CLASS | |
UAF$W_SALT |
UAI$_SALT | |
UAF$L_UIC |
Not applicable | |
Caution: Failure to synchronize multiple copies of the SYSUAF files properly may result in unexplained login failures and unauthorized system access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”. Reference: Appendix B, Building Common Files discusses creation and management of the various elements of an OpenVMS Cluster common SYSUAF.DAT authorization database. | ||
SYSUAFALT.DAT [required] |
The system alternate user authorization file. This file serves as a backup to SYSUAF.DAT and is enabled via the SYSUAFALT system parameter. When more than one copy of this file exists, all copies must be updated after any change to any authorization records in this file. Note: This file may not exist in all configurations. Caution: Failure to synchronize multiple copies of this file properly may result in unexplained login failures and unauthorized system access. | |
VMS$PASSWORD_HISTORY.DATA [recommended] |
The system password history database. It is maintained by the system password change facility. When more than one copy of this file exists, all copies should be updated after any password change. Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy. | |
VMSMAIL_PROFILE.DATA [recommended] |
The system mail database. This file is maintained by the OpenVMS Mail utility and contains mail profiles for all system users. Among the information contained in this file is the list of all mail forwarding addresses in use on the system. When more than one copy of this file exists, all copies should be updated after any changes to mail forwarding. Caution: Failure to synchronize multiple copies of this file properly may result in unauthorized disclosure of information. | |
VMS$PASSWORD_DICTIONARY.DATA [recommended] |
The system password dictionary. The system password dictionary is a list of English language words and phrases that are not legal for use as account passwords. When more than one copy of this file exists, all copies should be updated after any site-specific additions. Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy. | |
VMS$PASSWORD_POLICY.EXE [recommended] |
Any site-specific password filters. It is created and installed by the site-security administrator or system manager. When more than one copy of this file exists, all copies should be identical. Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy. Note: System managers can create this file as an image to enforce their local password policy. This is an architecture-specific image file that cannot be shared among different architecture types. |
5.7. Network Security
User authentication
On Cluster systems connected using IP, ensure that the cluster communications over insecure WAN links are encrypted and authenticated.
OpenVMS Cluster membership management
On Cluster systems connected using IP, isolate IP subnets that are used for cluster communication from the public internet using a secure gateway as shown in Figure 5.6, “Virtual Private Network for Protecting Cluster Traffic”.
Figure 5.6. Virtual Private Network for Protecting Cluster Traffic Using a security audit log file
OpenVMS Cluster system managers must also ensure consistency in the use of DECnet software for intracluster communication.
5.7.1. Mechanisms
Depending on the level of network security required, you might also want to consider how other security mechanisms, such as protocol encryption and decryption, can promote additional security protection across the cluster.
Reference: See the VSI OpenVMS Guide to System Security.
5.8. Coordinating System Files
IF you are setting up... |
THEN follow the procedures in... |
---|---|
A common-environment OpenVMS Cluster that consists of newly installed systems |
VSI OpenVMS System Manager's Manual to build these files. Because the files on new operating systems are empty except for the Digital-supplied accounts, very little coordination is necessary. |
An OpenVMS Cluster that will combine one or more computers that have been running with computer-specific files |
Appendix B, Building Common Files to create common copies of the files from the computer-specific files. |
5.8.1. Procedure
In a common-environment cluster with one common system disk, you use a common copy of each system file and place the files in the SYS$COMMON:[SYSEXE] directory on the common system disk or on a disk that is mounted by all cluster nodes. No further action is required.
To prepare a common user environment for an OpenVMS Cluster system that includes more than one common OpenVMS Integrity server system disk or more than one common OpenVMS Alpha system disk, you must coordinate the system files on those disks.
Disks holding common resources must be mounted early in the system startup procedure, such as in the SYLOGICALS.COM procedure.
You must ensure that the disks are mounted with each OpenVMS Cluster reboot.
Step | Action |
---|---|
1 |
Decide where to locate the SYSUAF.DAT and NETPROXY.DAT files. In a cluster with multiple system disks, system management is much easier if the common system files are located on a single disk that is not a system disk. |
2 |
Copy SYS$SYSTEM:SYSUAF.DAT and SYS$SYSTEM:NETPROXY.DAT to a location other than the system disk. |
3 |
Copy SYS$SYSTEM:RIGHTSLIST.DAT and SYS$SYSTEM:VMSMAIL_PROFILE.DATA to the same directory in which SYSUAF.DAT and NETPROXY.DAT reside. |
4 |
Edit the file SYS$COMMON:[SYSMGR]SYLOGICALS.COM on each system disk and define logical names that specify the location of the cluster common files. Example: If the files
will be located on $1$DGA16, define logical names as
follows:
$ DEFINE/SYSTEM/EXEC SYSUAF - $1$DGA16:[VMS$COMMON.SYSEXE]SYSUAF.DAT $ DEFINE/SYSTEM/EXEC NETPROXY - $1$DGA16:[VMS$COMMON.SYSEXE]NETPROXY.DAT $ DEFINE/SYSTEM/EXEC RIGHTSLIST - $1$DGA16:[VMS$COMMON.SYSEXE]RIGHTSLIST.DAT $ DEFINE/SYSTEM/EXEC VMSMAIL_PROFILE - $1$DGA16:[VMS$COMMON.SYSEXE]VMSMAIL_PROFILE.DATA $ DEFINE/SYSTEM/EXEC NETNODE_REMOTE - $1$DGA16:[VMS$COMMON.SYSEXE]NETNODE_REMOTE.DAT $ DEFINE/SYSTEM/EXEC NETNODE_UPDATE - $1$DGA16:[VMS$COMMON.SYSMGR]NETNODE_UPDATE.COM $ DEFINE/SYSTEM/EXEC QMAN$MASTER - $1$DGA16:[VMS$COMMON.SYSEXE] |
5 |
To ensure that the system disks are mounted correctly with
each reboot, follow these steps:
$ @SYS$SYSDEVICE:[VMS$COMMON.SYSMGR]CLU_MOUNT_DISK.COM $1$DGA16: volume-label |
6 |
When you are ready to start the queuing system, be sure you have moved the queue and journal files to a cluster-available disk. Any cluster common disk is a good choice if the disk has sufficient space. Enter the following
command:
$ START/QUEUE/MANAGER $1$DGA16:[VMS$COMMON.SYSEXE] |
5.8.2. Network Database Files
In OpenVMS Cluster systems on the LAN and in mixed-interconnect clusters, you must also coordinate the SYS$MANAGER:NETNODE_UPDATE.COM file, which is a file that contains all essential network configuration data for satellites. NETNODE_UPDATE.COM is updated each time you add or remove a satellite or change its Ethernet or FDDI hardware address. This file is discussed more thoroughly in Section 10.4.2, “Satellite Network Data”.
In OpenVMS Cluster systems configured with DECnet for OpenVMS software, you must also coordinate NETNODE_REMOTE.DAT, which is the remote node network database.
5.9. System Time on the Cluster
When a computer joins the cluster, the cluster attempts to set the joining computer's system time to the current time on the cluster. Although it is likely that the system time will be similar on each cluster computer, there is no assurance that the time will be set. Also, no attempt is made to ensure that the system times remain similar throughout the cluster. (For example, there is no protection against different computers having different clock rates).
An OpenVMS Cluster system spanning multiple time zones must use a single, clusterwide common time on all nodes. Use of a common time ensures timestamp consistency (for example, between applications, file-system instances) across the OpenVMS Cluster members.
5.9.1. Setting System Time
Use the SYSMAN command CONFIGURATION SET TIME to set the time across the cluster. This command issues warnings if the time on all nodes cannot be set within certain limits. Refer to the VSI OpenVMS System Manager's Manual for information about the SET TIME command.
Chapter 6. Cluster Storage Devices
One of the most important features of OpenVMS Cluster systems is the ability to provide access to devices and files across multiple systems.
In a traditional computing environment, a single system is directly attached to its storage subsystems. Even though the system may be networked with other systems, when the system is shut down, no other system on the network has access to its disks or any other devices attached to the system.
In an OpenVMS Cluster system, disks and tapes can be made accessible to one or more members. So, if one computer shuts down, the remaining computers still have access to the devices.
6.1. Data File Sharing
Integrity servers
Integrity servers and AlphaServer systems
AlphaServer systems
In addition, multiple systems that are permitted in the same OpenVMS Cluster system can write to a shared disk file simultaneously. It is this ability that allows multiple systems in an OpenVMS Cluster to share a single system disk; multiple systems can boot from the same system disk and share operating system files and utilities to save disk space and simplify system management.
Note
Tapes do not allow multiple systems to access a tape file simultaneously.
6.1.1. Access Methods
Depending on your business needs, you may want to restrict access to a particular device to the users on the computer that are directly connected (local) to the device. Alternatively, you may decide to set up a disk or tape as a served device so that any user on any OpenVMS Cluster computer can allocate and use it.
Method | Device Access | Comments | Illustrated in |
---|---|---|---|
Local |
Restricted to the computer that is directly connected to the device. |
Can be set up to be served to other systems. | |
Dual ported |
Using either of two physical ports, each of which can be connected to separate controllers. A dual-ported disk can survive the failure of a single controller by failing over to the other controller. |
As long as one of the controllers is available, the device is accessible by all systems in the cluster. | |
Shared |
Through a shared interconnect to multiple systems. |
Can be set up to be served to systems that are not on the shared interconnect. | |
Served |
Through a computer that has the MSCP or TMSCP server software loaded. |
MSCP and TMSCP serving are discussed in Section 6.3, “MSCP and TMSCP Served Disks and Tapes”. | |
Dual pathed |
Possible through more than one path. |
If one path fails, the device is accessed over the other path. Requires the use of allocation classes (described in Section 6.2.1, “Allocation Classes”to provide a unique, path-independent name.) | |
Note: The path to an individual disk may appear to be local from some nodes and served from others. |
6.1.2. Examples
When storage subsystems are connected directly to a specific system, the availability of the subsystem is lower due to the reliance on the host system. To increase the availability of these configurations, OpenVMS Cluster systems support dual porting, dual pathing, and MSCP and TMSCP serving.
Figure 6.1, “Dual-Ported Disks” shows a dual-ported configuration, in which the disks have independent connections to two separate computers. As long as one of the computers is available, the disk is accessible by the other systems in the cluster.

Note
Disks can be shadowed using Volume Shadowing for OpenVMS. The automatic recovery from system failure provided by dual porting and shadowing is transparent to users and does not require any operator intervention.
Figure 6.2, “Dual-Pathed Disks” shows a dual-pathed FC and Ethernet configuration. The disk devices, accessible through a shared SCSI interconnect, are MSCP served to the client nodes on the LAN.
Rule: A dual-pathed DSA disk cannot be used as a system disk for a directly connected CPU. Because a device can be on line to one controller at a time, only one of the server nodes can use its local connection to the device. The second server node accesses the device through the MSCP (or the TMSCP server). If the computer that is currently serving the device fails, the other computer detects the failure and fails the device over to its local connection. The device thereby remains available to the cluster.

The same device controller letter is generated and the same allocation class is specified on each computer, with the result that the device has the same name on both systems. (Section 6.2.1, “Allocation Classes” describes allocation classes).
Both computers are running the MSCP server for disks, the TMSCP server for tapes, or both.
Caution
Failure to observe these requirements can endanger data integrity.
You can set up HSG or HSV storage devices to be dual ported between two storage subsystems, as shown in Figure 6.3, “Configuration with Cluster-Accessible Devices”.

By design, HSG and HSV disks and tapes are directly accessible by all OpenVMS Cluster nodes that are connected to the same star coupler. Therefore, if the devices are dual ported, they are automatically dual pathed. Computers connected by FC can access a dual-ported HSG or HSV device by way of a path through either subsystem connected to the device. If one subsystem fails, access fails over to the other subsystem.
Note
To control the path that is taken during failover, you can specify a preferred path to force access to disks over a specific path. Section 6.1.3, “Specifying a Preferred Path” describes the preferred-path capability.
See the Guidelines for OpenVMS Cluster Configurations, for more information on FC storage devices.
6.1.3. Specifying a Preferred Path
For the first attempt to locate the disk and bring it on line with a DCL command MOUNT
For failover of an already mounted disk
In addition, you can initiate failover of a mounted disk to force the disk to the preferred path or to use load-balancing information for disks accessed by MSCP servers.
You can specify the preferred path by using the SET PREFERRED_PATH DCL command or by using the $QIO function (IO$_SETPRFPATH),with the P1 parameter containing the address of a counted ASCII string(.ASCIC). This string is the node name of the HSG or HSV, or of the OpenVMS system that is to be the preferred path.
Rule: The node name must match an existing node running the MSCP server that is known to the local node.
Reference: For more information about the use of the SET PREFERRED_PATH DCL command, refer to the VSI OpenVMS DCL Dictionary.
For more information about the use of the IO$_SETPRFPATH function, refer to the VSI OpenVMS I/O User's Reference Manual.
6.2. Naming OpenVMS Cluster Storage Devices
Note
The naming convention of Fibre Channel devices is documented in the Guidelines for OpenVMS Cluster Configurations. The naming of all other devices is described in this section.
dd represents the predefined code for the device type
c represents the predefined controller designation
u represents the unit number
For SCSI, the controller letter is assigned by OpenVMS, based on the system configuration. The unit number is determined by the SCSI bus ID and the logical unit number (LUN) of the device.
- If a device is attached to a single computer, the device name is extended to include the name of that computer:
node$ddcu
where node represents the SCS node name of the system on which the device resides.
- If a device is attached to multiple computers, the node name part of the device name is replaced by a dollar sign and a number (called a node or port allocation class, depending on usage),as follows:
$allocation-class$ddcu
SAS disks follow the device naming similar to that of SCSI devices, that is, Target-LUN numbering. So a disk on SAS target ID 1 and LUN 0 will be named as DKA100. For SAS tapes you can use the Fibre channel naming convention, that is, DGAtxx: The SYSGEN parameter SAS_NAMING can be used to use SCSI numbering in tapes also.
6.2.1. Allocation Classes
The purpose of allocation classes is to provide unique and unchanging device names. The device name is used by the OpenVMS Cluster distributed lock manager in conjunction with OpenVMS facilities (such as RMS and the XQP) to uniquely identify shared devices, files, and data.
Allocation classes are required in OpenVMS Cluster configurations where storage devices are accessible through multiple paths. Without the use of allocation classes, device names that relied on node names would change as access paths to the devices change.
Prior to OpenVMS Version 7.1, only one type of allocation class existed, which was node based. It was named allocation class. OpenVMS Version 7.1 introduced a second type, port allocation class, which is specific to a single interconnect and is assigned to all devices attached to that interconnect. Port allocation classes were originally designed for naming SCSI devices. Their use has been expanded to include additional devices types: floppy disks, PCI RAID controller disks, and IDE disks.
The use of port allocation classes is optional. They are designed to solve the device-naming and configuration conflicts that can occur in certain configurations, as described in Section 6.2.3, “Reasons for Using Port Allocation Classes”.
To differentiate between the earlier node-based allocation class and the newer port allocation class, the term node allocation classwas assigned to the earlier type.
Note
If SCSI devices are connected to multiple hosts and if port allocation classes are not used, then all nodes with direct access to the same multipathed devices must use the same nonzero node allocation class.
Multipathed MSCP controllers also have an allocation class parameter, which is set to match that of the connected nodes. (If the allocation class does not match, the devices attached to the nodes cannot be served).
6.2.2. Specifying Node Allocation Classes
A node allocation class can be assigned to computers, HSG or HSV controllers. The node allocation class is a numeric value from 1 to 255 that is assigned by the system manager.
The default node allocation class value is 0. A node allocation class value of 0 is appropriate only when serving a local, single-pathed disk. If a node allocation class of 0 is assigned, served devices are named using the node-name$device-name syntax, that is, the device name prefix reverts to the node name.
When serving satellites, the same nonzero node allocation class value must be assigned to the serving computers and controllers.
All cluster-accessible devices on computers with a nonzero node allocation class value must have unique names throughout the cluster. For example, if two computers have the same node allocation class value, it is invalid for both computers to have a local disk named DGA0 or a tape named MUA0. This also applies to HSG and HSV subsystems.
System managers provide node allocation classes separately for disks and tapes. The node allocation class for disks and the node allocation class for tapes can be different.
$disk-allocation-class$device-name $tape-allocation-class$device-name
Caution: Failure to set node allocation class values and device unit numbers correctly can endanger data integrity and cause locking conflicts that suspend normal cluster operations.
Figure 6.5, “Device Names in a Mixed-Interconnect Cluster” includes satellite nodes that access devices $1$DUA17 and $1$MUA12 through the JUPITR and NEPTUN computers. In this configuration, the computers JUPITR and NEPTUN require node allocation classes so that the satellite nodes are able to use consistent device names regardless of the access path to the devices.
Note: System management is usually simplified by using the same node allocation class value for all servers, HSG and HSV subsystems; you can arbitrarily choose a number between 1 and 255. Note, however, that to change a node allocation class value, you must shutdown and reboot the entire cluster (described in Section 8.6, “Post-configuration Tasks”). If you use a common node allocation class for computers and controllers, ensure that all devices have unique unit numbers.
6.2.2.1. Assigning Node Allocation Class Values on Computers
Step | Action |
---|---|
1 |
Edit the root directory [SYS
n.SYSEXE]MODPARAMS.DAT on each node
that boots from the system disk. The following example
shows a MODPARAMS.DAT file. The entries are hypothetical
and should be regarded as examples, not as suggestions
for specific parameter
settings.
! ! Site-specific AUTOGEN data file. In an OpenVMS Cluster ! where a common system disk is being used, this file ! should reside in SYS$SPECIFIC:[SYSEXE], not a common ! system directory. ! ! Add modifications that you want to make to AUTOGEN’s ! hardware configuration data, system parameter ! calculations, and page, swap, and dump file sizes ! to the bottom of this file. SCSNODE="NODE01" SCSSYSTEMID=99999 NISCS_LOAD_PEA0=1 VAXCLUSTER=2 MSCP_LOAD=1 MSCP_SERVE_ALL=1 ALLOCLASS=1 TAPE_ALLOCLASS=1 |
2 |
Invoke AUTOGEN to set the system parameter values:
$ @SYS$UPDATE:AUTOGEN start-phase end-phase |
3 |
Shut down and reboot the entire cluster in order for the new values to take effect. |
6.2.2.2. Node Allocation Class Example With a DSA Disk and Tape
Figure 6.4, “Disk and Tape Dual Pathed Between Computers ” shows a DSA disk and tape that are dual pathed between two computers.

URANUS and NEPTUN access the disk either locally or through the other computer's MSCP server.
When satellites ARIEL and OBERON access $1$DGA8, a path is made through either URANUS or NEPTUN.
If, for example, the node URANUS has been shut down, the satellites can access the devices through NEPTUN. When URANUS reboots, access is available through either URANUS or NEPTUN.
6.2.2.3. Node Allocation Class Example With Mixed Interconnects
Figure 6.5, “Device Names in a Mixed-Interconnect Cluster” shows how device names are typically specified in a mixed-interconnect cluster. This figure also shows how relevant system parameter values are set for each FC computer.

A disk and a tape are dual pathed to the HSG or HSV subsystems named VOYGR1 and VOYGR2; these subsystems are connected to JUPITR, SATURN, URANUS and NEPTUN through the star coupler.
The MSCP and TMSCP servers are loaded on JUPITR and NEPTUN (MSCP_LOAD = 1, TMSCP_LOAD = 1) and the ALLOCLASS and TAPE_ALLOCLASS parameters are set to the same value (1) on these computers and on both HSG or HSV subsystems.
Note: For optimal availability, two or more FC connected computers can serve HSG or HSV devices to the cluster.
6.2.2.4. Node Allocation Classes and RAID Array 210 and 230 Devices
If you have RAID devices connected to StorageWorks RAID Array 210 or 230 subsystems, you might experience device-naming problems when running in a cluster environment if nonzero node allocation classes are used. In this case, the RAID devices will be named $ n$DRcu, where n is the (nonzero) node allocation class, c is the controller letter, and u is the unit number.
If multiple nodes in the cluster have the same (nonzero) node allocation class and these same nodes have RAID controllers, then RAID devices that are distinct might be given the same name (for example, $1$DRA0). This problem can lead to data corruption.
To prevent such problems, use the DR_UNIT_BASE system parameter, which causes the DR devices to be numbered sequentially, starting with the DR_UNIT_BASE value that you specify. For example, if the node allocation class is $1, the controller letter is A, and you set DR_UNIT_BASE on one cluster member to 10, the first device name generated by the RAID controller will be $1$DRA10, followed by $1$DRA11, $1$DRA12, and so forth.
To ensure unique DR device names, set the DR_UNIT_BASE number on each cluster member so that the resulting device numbers do not overlap. For example, you can set DR_UNIT_BASE on three cluster members to 10, 20, and 30respectively. As long as each cluster member has 10 or fewer devices, the DR device numbers will be unique.
6.2.3. Reasons for Using Port Allocation Classes
When the node allocation class is nonzero, it becomes the device name prefix for all attached devices, whether the devices are on a shared interconnect or not. To ensure unique names within a cluster, it is necessary for the ddcu part of the disk device name (for example, DKB0) to be unique within an allocation class, even if the device is on a private bus.
This constraint is relatively easy to overcome for DIGITAL Storage Architecture(DSA) devices, because a system manager can select from a large unit number space to ensure uniqueness. The constraint is more difficult to manage for other device types, such as SCSI devices whose controller letter and unit number are determined by the hardware configuration.
For example, in the configuration shown in Figure 6.6, “SCSI Device Names Using a Node Allocation Class”, each system has a private SCSI bus with adapter letter A. To obtain unique names, the unit numbers must be different. This constrains the configuration to a maximum of 8 devices on the two buses (or 16 if wide addressing can be used on one or more of the buses). This can result in empty StorageWorks drive bays and in a reduction of the system's maximum storage capacity.

6.2.3.1. Constraint of the SCSI Controller Letter in Device Names
The SCSI device name is determined in part by the SCSI controller through which the device is accessed (for example, B in DKB n). Therefore, to ensure that each node uses the same name for each device, all SCSI controllers attached to a shared SCSI bus must have the same OpenVMS device name. In Figure 6.6, “SCSI Device Names Using a Node Allocation Class”, each host is attached to the shared SCSI bus by controller PKB.
Built-in SCSI controllers that are not supported in SCSI clusters
Long internal cables that make some controllers inappropriate for SCSI clusters
6.2.3.2. Constraints Removed by Port Allocation Classes
A system manager can specify an allocation class value that is specific to a port rather than nodewide.
When a port has a nonzero port allocation class, the controller letter in the device name that is accessed through that port is always the letter A.
Using port allocation classes for naming SCSI, IDE, floppy disk, and PCI RAID controller devices removes the configuration constraints described in Section 6.2.2.4, “Node Allocation Classes and RAID Array 210 and 230 Devices”, in Section 6.2.3, “Reasons for Using Port Allocation Classes”, and in Section 6.2.3.1, “Constraint of the SCSI Controller Letter in Device Names”. You do not need to use the DR_UNIT_BASE system parameter recommended in Section 6.2.2.4, “Node Allocation Classes and RAID Array 210 and 230 Devices”. Furthermore, each bus can be given its own unique allocation class value, so the ddcu part of the disk device name (for example, DKB0) does not need to be unique across buses. Moreover, controllers with different device names can be attached to the same bus, because the disk device names no longer depend on the controller letter.
Attach an adapter with a name (PKA) that differs from the name of the other adapters (PKB) attached to the shared SCSI interconnect, as long as that port has the same port allocation class (116 in this example).
Use two disks with the same controller name and number (DKA300) be cause each disk is attached to a SCSI interconnect that is not shared.

6.2.4. Specifying Port Allocation Classes
A port allocation class is a designation for all ports attached to a single interconnect. It replaces the node allocation class in the device name.
Port allocation classes of 1 to 32767 for devices attached to a multihost interconnect or a single-host interconnect, if desired
Port allocation class 0 for devices attached to a single-host interconnect
Port allocation class -1 when no port allocation class is in effect
Each type has its own naming rules.
6.2.4.1. Port Allocation Classes for Devices Attached to a Multi-Host Interconnect
The valid range of port allocation classes is 1 through 32767.
When using port allocation classes, the controller letter in the device name is always A, regardless of the actual controller letter. The $GETDVI item code DVI$_DISPLAY_DEVNAM displays the actual port name.
Note that it is now more important to use fully specified names (for example, $101$DKA100 or ABLE$DKA100) rather than abbreviated names (such as DK100), because a system can have multiple DKA100 disks.
Each port allocation class must be unique within a cluster.
A port allocation class cannot duplicate the value of another node's tape or disk node allocation class.
Each node for which MSCP serves a device should have the same nonzero allocation class value.
Device Name |
Description |
---|---|
$101$DKA0 |
The port allocation class is 101; DK represents the disk device category, A is the controller name, and 0 is the unit number. |
$147$DKA0 |
The port allocation class is 147; DK represents the disk device category, A is the controller name, and 0 is the unit number. |
6.2.4.2. Port Allocation Class 0 for Devices Attached to a Single-Host Interconnect
Port allocation class 0 does not become part of the device name. Instead, the name of the node to which the device is attached becomes the first part of the device name.
The controller letter in the device name remains the designation of the controller to which the device is attached. (It is not changed to A as it is for port allocation classes greater than zero.)
Device Name |
Description |
---|---|
ABLE$DKD100 |
ABLE is the name of the node to which the device is attached. D is the designation of the controller to which it is attached, not A as it is for port allocation classes with a nonzero class. The unit number of this device is 100. The port allocation class of $0$ is not included in the device name. |
BAKER$DKC200 |
BAKER is the name of the node to which the device is attached, C is the designation of the controller to which it is attached, and 200 is the unit number. The port allocation class of $0$ is not included in the device name. |
6.2.4.3. Port Allocation Class -1
The designation of port allocation class -1 means that a port allocation class is not being used. Instead, a node allocation class is used. The controller letter remains its predefined designation. (It is assigned by OpenVMS, based on the system configuration. It is not affected by a node allocation class).
6.2.4.4. How to Implement Port Allocation Classes
Port allocation classes were introduced in OpenVMS Alpha Version 7.1 with support in OpenVMS VAX. VAX computers can serve disks connected to Alpha systems that use port allocation classes in their names.
Enable the use of port allocation classes.
Assign one or more port allocation classes.
At a minimum, reboot the nodes on the shared SCSI bus.
Enabling the Use of Port Allocation Classes
To enable the use of port allocation classes, you must set a new SYSGEN parameter DEVICE_NAMING to 1. The default setting for this parameter is zero. In addition, the SCSSYSTEMIDH system parameter must be set to zero. Check to make sure that it is.
Assigning Port Allocation Classes
You can assign one or more port allocation classes with the OpenVMS Cluster configuration procedure, CLUSTER_CONFIG.COM (or CLUSTER_CONFIG_LAN.COM).
If it is not possible to use CLUSTER_CONFIG.COM or CLUSTER_CONFIG_LAN.COM to assign port allocation classes (for example, if you are booting a private system disk into an existing cluster), you can use the new SYSBOOT SET/CLASS command.
SYSBOOT> SET/CLASS PKB 152
The SYSINIT process ensures that this new name is used in successive boots.
SYSBOOT> SET/CLASS PKB
The mapping of ports to allocation classes is stored in SYS$SYSTEM:SYS$DEVICES.DAT, a standard text file. You use the CLUSTER_CONFIG.COM (or CLUSTER_CONFIG_LAN.COM) command procedure or, in special cases, SYSBOOT to change SYS$DEVICES.DAT.
6.2.4.5. Clusterwide Reboot Requirements for SCSI Interconnects
Changing a device's allocation class changes the device name. A clusterwide reboot ensures that all nodes see the device under its new name, which in turn means that the normal device and file locks remain consistent.
Dismount the devices whose names have changed from all nodes.
This is not always possible. In particular, you cannot dismount a disk on nodes where it is the system disk. If the disk is not dismounted, a subsequent attempt to mount the same disk using the new device name will fail with the following error:%MOUNT-F-VOLALRMNT, another volume of same label already mounted
Therefore, you must reboot any node that cannot dismount the disk.
Reboot all nodes connected to the SCSI bus.
Before you reboot any of these nodes, make sure the disks on the SCSI bus are dismounted on the nodes not rebooting.Note
OpenVMS ensures that a node cannot boot if the result is a SCSI bus with naming different from another node already accessing the same bus. (This check is independent of the dismount check in step 1).
After the nodes that are connected to the SCSI bus reboot, the device exists with its new name.
Mount the devices systemwide or clusterwide.
If no other node has the disk mounted under the old name, you can mount the disk systemwide or clusterwide using its new name. The new device name will be seen on all nodes running compatible software, and these nodes can also mount the disk and access it normally.
Nodes that have not rebooted still see the old device name as well as the new device name. However, the old device name cannot be used; the device, when accessed by the old name, is off line. The old name persists until the node reboots.
6.3. MSCP and TMSCP Served Disks and Tapes
The MSCP server and the TMSCP server make locally connected disks and tapes available to all cluster members. Locally connected disks and tapes are not automatically cluster accessible. Access to these devices is restricted to the local computer unless you explicitly set them up as cluster accessible using the MSCP server for disks or the TMSCP server for tapes.
6.3.1. Enabling Servers
Loaded on the local computer, as described in Table 6.4, “MSCP_LOAD and TMSCP_LOAD Parameter Settings”
Made functional by setting the MSCP and TMSCP system parameters, as described in Table 6.5, “MSCP_SERVE_ALL and TMSCP_SERVE_ALL Parameter Settings”
Parameter |
Value |
Meaning |
---|---|---|
MSCP_LOAD |
0 |
Do not load the MSCP_SERVER. This is the default. |
1 |
Load the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter using the default CPU load capacity. | |
>1 |
Load the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter. Use the MSCP_LOAD value as the CPU load capacity. | |
TMSCP_LOAD |
0 |
Do not load the TMSCP server and do not serve any tapes (default value). |
1 |
Load the TMSCP server and serve all available tapes, including all local tapes and all multihost tapes with a matching TAPE_ALLOCLASS value. |
Table 6.5, “MSCP_SERVE_ALL and TMSCP_SERVE_ALL Parameter Settings” summarizes the system parameter values you can specify for MSCP_SERVE_ALL and TMSCP_SERVE_ALL to configure the MSCP and TMSCP servers. Initial values are determined by your responses when you execute the installation or upgrade procedure or when you execute the CLUSTER_CONFIG.COM command procedure described in Chapter 8, Configuring an OpenVMS Cluster System to set up your configuration.
Note
In a mixed-version cluster that includes any systems running OpenVMS Version 7.1- x or earlier, serving all available disks is restricted to serving all disks whose allocation class matches the system's node allocation class (pre-Version 7.2 meaning).To specify this type of serving, use the value 9 (which sets bit 0 and bit 3).
Parameter |
Bit |
Value When Set |
Meaning |
---|---|---|---|
MSCP_SERVE_ALL |
0 |
1 |
Serve all available disks (locally attached and those connected to HS x and DSSI controllers). Disks with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are also served if bit 3 is not set. |
1 |
2 |
Serve locally attached (non-HS x and non-DSSI)disks. The server does not monitor its I/O traffic and does not participate in load balancing. | |
2 |
4 |
Serve the system disk. This is the default setting. This setting is important when other nodes in the cluster rely on this system being able to serve its system disk. This setting prevents obscure contention problems that can occur when a system attempts to complete I/O to a remote system disk whose system has failed. For more information, see Section 6.3.1.1, “Serving the System Disk”. | |
3 |
8 |
Restrict the serving specified by bit 0. All disks except those with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are served. This is pre-Version 7.2 behavior. If your cluster includes systems running Open 7.1- x or earlier, and you want to serve all available disks, you must specify 9, the result of setting this bit and bit 0. | |
4 |
15 |
By default, the bit 4 is not set, hence the DUDRIVER will accept the devices with unit number greater than 9999. On the client side, if bit 4 is set (10000 binary) in the MSCP_SERVE_ALL parameter, the client will reject devices with unit number greater than 9999 and retains the earlier behavior. | |
TMSCP_SERVE_ALL |
0 |
1 |
Serve all available tapes (locally attached and those connected to HS x and DSSI controllers). Tapes with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are also served if bit 3 is not set. |
1 |
2 |
Serve locally attached (non-HS x and non-DSSI) tapes. | |
3 |
8 |
Restrict the serving specified by bit 0. Serve all tapes except those with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter). This is pre-Version 7.2 behavior. If your cluster includes systems running OpenVMS Version 7.1- x or earlier, and you want to serve all available tapes, you must specify 9, the result of setting this bit and bit 0. | |
4 |
15 |
By default, the bit 4 is not set, hence the TUDRIVER will accept the devices with unit number greater than 9999. On the client side, if bit 4 is set (10000 binary) in the TMSCP_SERVE_ALL parameter, the client will reject devices with unit number greater than 9999 and retains the earlier behavior. |
Value |
Description |
---|---|
0 |
Do not serve any disks (tapes). This is the default. |
1 |
Serve all available disks (tapes). |
2 |
Serve only locally attached (non-HS x and non-DSSI) disks (tapes). |
6.3.1.1. Serving the System Disk
Setting bit 2 of the MSCP_SERVE_ALL system parameter to serve the system disk is important when other nodes in the cluster rely on this system being able to serve its system disk. This setting prevents obscure contention problems that can occur when a system attempts to complete I/O to are mote system disk whose system has failed.
The MSCP_SERVE_ALL setting is changed to disable serving when the system reboots.
The serving system crashes.
The client system that was executing I/O to the serving system's system disk is holding locks on resources of that system disk.
The client system starts mount verification.
The serving system attempts to boot but cannot because of the locks held on its system disk by the client system.
The client's mount verification process times out after a period of time set by the MVTIMEOUT system parameter, and the client system releases the locks. The time period could be several hours.
The serving system is able to reboot.
6.3.1.2. Setting the MSCP and TMSCP System Parameters
Specify appropriate values for these parameters in a computer's MODPARAMS.DAT file and then run AUTOGEN.
Run the CLUSTER_CONFIG.COM or the CLUSTER_CONFIG_LAN.COM procedure, as appropriate, and choose the CHANGE option to perform these operations for disks and tapes.
With either method, the served devices become accessible when the serving computer reboots. Further, the servers automatically serve any suitable device that is added to the system later. For example, if new drives are attached to an HSC subsystem, the devices are dynamically configured.
Note: The SCSI retention command modifier is not supported by the TMSCP server. Retention operations should be performed from the node serving the tape.
6.4. MSCP I/O Load Balancing
Faster I/O response
Balanced work load among the members of an OpenVMS Cluster
Two types of MSCP I/O load balancing are provided by OpenVMS Cluster software: static and dynamic. Static load balancing occurs on Integrity servers and Alpha systems and are based on the load capacity ratings of the server systems.
6.4.1. Load Capacity
The load capacity ratings for Integrity servers and Alpha systems are predetermined by VSI. These ratings are used in the calculation of the available serving capacity for MSCP static and dynamic load balancing. You can override these default settings by specifying a different load capacity with the MSCP_LOAD parameter.
Note that the MSCP server load-capacity values (either the default value or the value you specify with MSCP_LOAD) are estimates used by the load-balancing feature. They cannot change the actual MSCP serving capacity of a system.
A system's MSCP serving capacity depends on many factors including its power, the performance of its LAN adapter, and the impact of other processing loads. The available serving capacity, which is calculated by each MSCP server as described in Section 6.4.2, “Available Serving Capacity”, is used solely to bias the selection process when a client system (for example, a satellite) chooses which server system to use when accessing a served disk.
6.4.2. Available Serving Capacity
The load-capacity ratings are used by each MSCP server to calculate its available serving capacity.
Step |
Calculation |
---|---|
1 |
Each MSCP server counts the read and write requests sent to it and periodically converts this value to requests per second. |
2 |
Each MSCP server subtracts its requests per second from its load capacity to compute its available serving capacity. |
6.4.3. Static Load Balancing
MSCP servers periodically send their available serving capacities to the MSCP class driver(DUDRIVER). When a disk is mounted or one fails over, DUDRIVER assigns the server with the highest available serving capacity to it. (TMSCP servers do not perform this monitoring function.) This initial assignment is called static load balancing.
6.4.4. Overriding MSCP I/O Load Balancing for Special Purposes
In some configurations, you may want to designate one or more systems in your cluster as the primary I/O servers and restrict I/O traffic on other systems. You can accomplish these goals by overriding the default load-capacity ratings used by the MSCP server. For example, if your cluster consists of two Alpha systems and one VAX 6000-400 system and you want to reduce the MSCP served I/O traffic to the VAX, you can assign the VAX a low MSCP_LOAD value, such as 50. Because the two Alpha systems each start with a load-capacity rating of 340 and the VAX now starts with a load-capacity rating of 50, the MSCP served satellites will direct most of the I/O traffic to the Alpha systems.
6.5. Managing Cluster Disks With the Mount Utility
For locally connected disks to be accessible to other nodes in the cluster, the MSCP server software must be loaded on the computer to which the disks are connected (see Section 6.3.1, “Enabling Servers”). Further, each disk must be mounted with the Mount utility, using the appropriate qualifier: /CLUSTER, /SYSTEM, or /GROUP. Mounting multiple disks can be automated with command procedures; a sample command procedure, MSCPMOUNT.COM, is provided in the SYS$EXAMPLES directory on your system.
The Mount utility also provides other qualifiers that determine whether a disk is automatically rebuilt during a remount operation. Different rebuilding techniques are recommended for data and system disks.
This section describes how to use the Mount utility for these purposes.
6.5.1. Mounting Cluster Disks
IF... |
THEN... |
---|---|
At system startup | |
The disk is attached to a single system and is to be made available to all other nodes in the cluster. |
Use MOUNT/CLUSTER device-name on the computer to which the disk is to be mounted. The disk is mounted on every computer that is active in the cluster at the time the command executes. First, the disk is mounted locally. Then, if the mount operation succeeds, the disk is mounted on other nodes in the cluster. |
The computer has no disks directly attached to it. |
Use MOUNT/SYSTEM device-name on the computer for each disk the computer needs to access. The disks can be attached to a single system or shared disks that are accessed by an HS xcontroller. Then, if the mount operation succeeds, the disk is mounted on the computer joining the cluster. |
When the system is running | |
You want to add a disk. |
Use MOUNT/CLUSTER device-name on the computer to which the disk is to be mounted. The disk is mounted on every computer that is active in the cluster at the time the command executes. First, the disk is mounted locally. Then, if the mount operation succeeds, the disk is mounted on other nodes in the cluster. |
To ensure disks are mounted whenever possible, regardless of the sequence that systems in the cluster boot (or shut down), startup command procedures should use MOUNT/CLUSTER and MOUNT/SYSTEM as described in the preceding table.
Note: Only system or group disks can be mounted across the cluster or on a subset of the cluster members. If you specify MOUNT/CLUSTER without the /SYSTEM or /GROUP qualifier, /SYSTEM is assumed. Also note that each cluster disk mounted with the /SYSTEM or /GROUP qualifier must have a unique volume label.
6.5.2. Examples of Mounting Shared Disks
$ MOUNT/CLUSTER/NOASSIST $1$DUA4: COMPANYDOCS
$ MOUNT/SYSTEM/NOASSIST $1$DUA4: COMPANYDOCS
To mount the disk at startup time, include the MOUNT command either in a common command procedure that is invoked at startup time or in the computer-specific startup command file.
Note: The /NOASSIST qualifier is used in command procedures that are designed to make several attempts to mount disks. The disks may be temporarily offline or otherwise not available for mounting. If, after several attempts, the disk cannot be mounted, the procedure continues. The /ASSIST qualifier, which is the default, causes a command procedure to stop and query the operator if a disk cannot be mounted immediately.
6.5.3. Mounting Cluster Disks With Command Procedures
As a separate file specific to each computer in the cluster by making copies of the common procedure and storing them as separate files
As a common computer-independent file on a shared disk
With either method, each computer can invoke the common procedure from the site-specific SYSTARTUP procedure.
Example: The MSCPMOUNT.COM file in the SYS$EXAMPLES directory on your system is a sample command procedure that contains commands typically used to mount cluster disks. The example includes comments explaining each phase of the procedure.
6.5.4. Disk Rebuild Operation
To minimize disk I/O operations (and thus improve performance) when files are created or extended, the OpenVMS file system maintains a cache of preallocated file headers and disk blocks.
If a disk is dismounted improperly—for example, if a system fails or is removed from a cluster without running SYS$SYSTEM:SHUTDOWN.COM—this preallocated space becomes temporarily unavailable. When the disk is remounted, MOUNT scans the disk to recover the space. This is called a disk rebuild operation.
6.5.5. Rebuilding Cluster Disks
On a nonclustered computer, the MOUNT scan operation for recovering preallocated space merely prolongs the boot process. In an OpenVMS Cluster system, however, this operation can degrade response time for all user processes in the cluster. While the scan is in progress on a particular disk, most activity on that disk is blocked.
Note: User processes that attempt to read or write to files on the disk can experience delays of several minutes or longer, especially if the disk contains a large number of files or has many users.
Because the rebuild operation can delay access to disks during the startup of any OpenVMS Cluster computer, VSI recommends that procedures for mounting cluster disks use the /NOREBUILD qualifier. When MOUNT/NOREBUILD is specified, disks are not scanned to recover lost space, and users experience minimal delays while computers are mounting disks.
Reference: Section 6.5.6, “Rebuilding System Disks” provides information about rebuilding system disks. Section 9.7.1, “Avoiding Disk Rebuilds ” provides more information about disk rebuilds and system-disk throughput techniques.
6.5.6. Rebuilding System Disks
Rebuilding system disks is especially critical because most system activity requires access to a system disk. When a system disk rebuild is in progress, very little activity is possible on any computer that uses that disk.
Setting |
Comments |
---|---|
ACP_REBLDSYSD parameter should be set to 0 on satellites. |
This setting prevents satellites from rebuilding a system disk when it is mounted early in the boot sequence and eliminates delays caused by such a rebuild when satellites join the cluster. |
ACP_REBLDSYSD should be set to the default value of 1 on boot servers, and procedures that mount disks on the boot servers should use the /REBUILD qualifier. |
While these measures can make boot server rebooting more noticeable, they ensure that system disk space is available after an unexpected shutdown. |
Once the cluster is up and running, system managers can submit a batch procedure that executes SET VOLUME/REBUILD commands to recover lost disk space. Such procedures can run at a time when users would not be inconvenienced by the blocked access to disks (for example, between midnight and 6 a.m. each day). Because the SET VOLUME/REBUILD command determines whether a rebuild is needed, the procedures can execute the command for each disk that is usually mounted.
Powerful computers
Computers that have direct access to the volume to be rebuilt
Moreover, several such procedures, each of which rebuilds a different set of disks, can be executed simultaneously.
Disks are mounted with the MOUNT/NOREBUILD command.
The ACP_REBLDSYSD system parameter is set to 0.
Failure to rebuild disk volumes can result in a loss of free space and in subsequent failures of applications to create or extend files.
6.6. Shadowing Disks Across an OpenVMS Cluster
Volume shadowing (sometimes referred to as disk mirroring) achieves high data availability by duplicating data on multiple disks. If one disk fails, the remaining disk or disks can continue to service application and user I/O requests.
6.6.1. Purpose
Volume Shadowing for OpenVMS software provides data availability across the full range of OpenVMS configurations—from single nodes to large OpenVMS Cluster systems—so you can provide data availability where you need it most.
Volume Shadowing for OpenVMS software is an implementation of RAID 1 (redundant arrays of independent disks) technology. Volume Shadowing for OpenVMS prevents a disk device failure from interrupting system and application operations. By duplicating data on multiple disks, volume shadowing transparently prevents your storage subsystems from becoming a single point of failure because of media deterioration, communication path failure, or controller or device failure.
6.6.2. Shadow Sets
You can mount up to six compatible disk volumes to form a shadow set. Figure 6.8, “Shadow Set With Three Members” shows three compatible disk volumes used to form a shadow set. Each disk in the shadow set is known as a shadow set member. Volume Shadowing for OpenVMS logically binds the shadow set devices together and represents them as a single virtual device called a virtual unit. This means that the multiple members of the shadow set, represented by the virtual unit, appear to operating systems and users as a single, highly available disk.

6.6.3. I/O Capabilities
Applications and users read and write data to and from a shadow set using the same commands and program language syntax and semantics that are used for nonshadowed I/O operations. System managers manage and monitor shadow sets using the same commands and utilities they use for nonshadowed disks. The only difference is that access is through the virtual unit, not to individual devices.
Reference: VSI OpenVMS Volume Shadowing Guide describes the shadowing product capabilities in detail.
6.6.4. Supported Devices
All MSCP compliant DSA drives
All SAS devices
All StorageWorks SCSI disks and controllers, and some third-party SCSI devices that implement READL (read long) and WRITEL (write long) commands and use the SCSI disk driver (DKDRIVER)
Restriction: SCSI disks that do not support READL and WRITEL are restricted because these disks do not support the shadowing data repair (disk bad-block errors) capability. Thus, using unsupported SCSI disks can cause members to be removed from the shadow set.
You can shadow data disks and system disks. Thus, a system disk need not be a single point of failure for any system that boots from that disk. System disk shadowing becomes especially important for OpenVMS Cluster systems that use a common system disk from which multiple computers boot.
Volume Shadowing for OpenVMS does not support the shadowing of quorum disks. This is because volume shadowing makes use of the OpenVMS distributed lock manager, and the quorum disk must be utilized before locking is enabled.
There are no restrictions on the location of shadow set members beyond the valid disk configurations defined in the Volume Shadowing for OpenVMS Software Product Description.
6.6.5. Shadow Set Limits
You can mount a default maximum of 500 shadow sets (each having one to six members) in a standalone system or OpenVMS Cluster system. If more than 500 shadow sets are required, the SYSGEN parameter SHADOW_MAX_UNIT must be increased. The number of shadow sets supported is independent of controller and device types. The shadow sets can be mounted as public or private volumes.
For any changes to these limits, consult the Volume Shadowing for OpenVMS Software Product Description.
6.6.6. Distributing Shadowed Disks
The controller-independent design of shadowing allows you to manage shadow sets regardless of their controller connection or location in the OpenVMS Cluster system and helps provide improved data availability and very flexible configurations.
For clusterwide shadowing, members can be located anywhere in an OpenVMS Cluster system and served by MSCP servers across any supported OpenVMS Cluster interconnect.
Figure 6.9, “Shadow Sets Accessed Through the MSCP Server” shows how shadow set member units are on line to local controllers located on different nodes. In the figure, a disk volume is local to each of the nodes ATABOY and ATAGRL. The MSCP server provides access to the shadow set members over the LAN or IP network. Even though the disk volumes are local to different nodes, the disks are members of the same shadow set. A member unit that is local to one node can be accessed by the remote node over the MSCP server.

For shadow sets that are mounted on an OpenVMS Cluster system, mounting or dismounting a shadow set on one node in the cluster does not affect applications or user functions executing on other nodes in the system. For example, you can dismount the virtual unit from one node in an OpenVMS Cluster system and leave the shadow set operational on the remaining nodes on which it is mounted.
If an individual disk volume is already mounted as a member of an active shadow set, the disk volume cannot be mounted as a standalone disk on another node at the same time.
- System disks can be shadowed. All nodes booting from shadowed system disks must:
Have a Volume Shadowing for OpenVMS license.
Set shadowing system parameters to enable shadowing and specify the system disk virtual unit number.
Chapter 7. Setting Up and Managing Cluster Queues
This chapter discusses queuing topics specific to OpenVMS Cluster systems. Because queues in an OpenVMS Cluster system are established and controlled with the same commands used to manage queues on a standalone computer, the discussions in this chapter assume some knowledge of queue management on a standalone system, as described in the VSI OpenVMS System Manager's Manual.
Note
See the VSI OpenVMS System Manager's Manual for information about queuing compatibility.
7.1. Introduction
Users can submit jobs to any queue in the OpenVMS Cluster system, regardless of the processor on which the job will actually execute. Generic queues can balance the work load among the available processors.
The system manager can use one or several queue managers to manage batch and print queues for an entire OpenVMS Cluster system. Although a single queue manager is sufficient for most systems, multiple queue managers can be useful for distributing the batch and print work load across nodes in the cluster.
Once the batch and print queue characteristics are set up, the system manager can rely on the distributed queue manager to make queues available across the cluster.
7.2. Controlling Queue Availability
WHEN... |
THEN... |
Comments |
---|---|---|
The node on which the queue manager is running leaves the OpenVMS Cluster system. |
The queue manager automatically fails over to another node. |
This failover occurs transparently to users on the system. |
Nodes are added to the cluster. |
The queue manager automatically serves the new nodes. |
The system manager does not need to enter a command explicitly to start queuing on the new node. |
The OpenVMS Cluster system reboots. |
The queuing system automatically restarts by default. |
Thus, you do not have to include commands in your startup command procedure for queuing. |
The operating system automatically restores the queuing system with the parameters defined in the queuing database. |
This is because when you start the queuing system, the characteristics you define are stored in a queue database. |
To control queues, the queue manager maintains a clusterwide queue database that stores information about queues and jobs. Whether you use one or several queue managers, only one queue database is shared across the cluster. Keeping the information for all processes in one database allows jobs submitted from any computer to execute on any queue (provided that the necessary mass storage devices are accessible).
7.3. Starting a Queue Manager and Creating the Queue Database
You start up a queue manager using the START/QUEUE/MANAGER command as you would on a standalone computer. However, in an OpenVMS Cluster system, you can also provide a failover list and a unique name for the queue manager. The /NEW_VERSION qualifier creates a new queue database.
$ START/QUEUE/MANAGER/NEW_VERSION/ON=(GEM,STONE,*)
Command |
Function |
---|---|
START/QUEUE/MANAGER |
Creates a single, clusterwide queue manager named SYS$QUEUE_MANAGER. |
/NEW_VERSION |
Creates a new queue database in SYS$COMMON:[SYSEXE] that
consists of the following three files:
Rule: Use the /NEW_VERSION qualifier only on the first invocation of the queue manager or if you want to create a new queue database. |
/ON= (node-list) [optional] |
Specifies an ordered list of nodes that can claim the queue
manager if the node running the queue manager should exit the
cluster. In the example:
|
/NAME_OF_MANAGER [optional] |
Allows you to assign a unique name to the queue manager.
Unique queue manager names are necessary if you run multiple
queue managers. For example, using the /NAME_OF_MANAGER
qualifier causes queue and journal files to be created using the
queue manager name instead of the default name
SYS$QUEUE_MANAGER. For example, adding the
/NAME_OF_MANAGER=PRINT_MANAGER qualifier command creates these files:
|
Rules for OpenVMS Cluster systems with
multiple system disks:
|
7.4. Starting Additional Queue Managers
Running multiple queue managers balances the work load by distributing batch and print jobs across the cluster. For example, you might create separate queue managers for batch and print queues in clusters with CPU or memory shortages. This allows the batch queue manager to run on one node while the print queue manager runs on a different node.
7.4.1. Command Format
$ START/QUEUE/MANAGER/ADD/NAME_OF_MANAGER=BATCH_MANAGER
7.4.2. Database Files
name_of_manager.QMAN$QUEUES
name_of_manager.QMAN$JOURNAL
By default, the queue database and its files are located in SYS$COMMON:[SYSEXE]. If you want to relocate the queue database files, refer to the instructions in Section 7.6, “Moving Queue Database Files”.
7.5. Stopping the Queuing System
When you enter the STOP/QUEUE/MANAGER/CLUSTER command, the queue manager remains stopped, and requests for queuing are denied until you enter the START/QUEUE/MANAGER command (without the /NEW_VERSION qualifier).
$ STOP/QUEUE/MANAGER/CLUSTER/NAME_OF_MANAGER=PRINT_MANAGER
Rule: You must include the /CLUSTER qualifier on the command line whether or not the queue manager is running on an OpenVMS Cluster system. If you omit the/CLUSTER qualifier, the command stops all queues on the default node without stopping the queue manager. (This has the same effect as entering the STOP/QUEUE/ON_NODE command).
7.6. Moving Queue Database Files
The files in the queue database can be relocated from the default location of SYS$COMMON:[SYSEXE] to any disk that is mounted clusterwide or that is accessible to the computers participating in the clusterwide queue scheme. For example, you can enhance system performance by locating the database on a shared disk that has a low level of activity.
7.6.1. Location Guidelines
The master file QMAN$MASTER can be in a location separate from the queue and journal files, but the queue and journal files must be kept together in the same directory. The queue and journal files for one queue manager can be separate from those of other queue managers.
The directory you specify must be available to all nodes in the cluster. If the directory specification is a concealed logical name, it must be defined identically in the SYS$COMMON:SYLOGICALS.COM startup command procedure on every node in the cluster.
Reference: The VSI OpenVMS System Manager's Manual contains complete information about creating or relocating the queue database files. See also Section 7.12, “Using a Common Command Procedure” for a sample common procedure that sets up an OpenVMS Cluster batch and print system.
7.7. Setting Up Print Queues
Which print queues you want to establish on each computer
Whether to set up any clusterwide generic queues to distribute print job processing across the cluster
Whether to set up auto start queues for availability or improved startup time
Once you determine the appropriate strategy for your cluster, you can create your queues. Figure 7.1, “Sample Printer Configuration” shows the printer configuration for a cluster consisting of the active computers JUPITR, SATURN, and URANUS.

7.7.1. Creating a Queue
You set up OpenVMS Cluster print queues using the same method that you would use for a standalone computer. However, in an OpenVMS Cluster system, you must provide a unique name for each queue you create.
7.7.2. Command Format
$ INITIALIZE/QUEUE/ON=node-name::device[/START][/NAME_OF_MANAGER=
name-of-manager] queue-name
Qualifier |
Description |
---|---|
/ON |
Specifies the computer and printer to which the queue is assigned. If you specify the /START qualifier, the queue is started. |
/NAME_OF_MANAGER |
If you are running multiple queue managers, you should also specify the queue manager with the qualifier. |
7.7.3. Ensuring Queue Availability
You can also use the auto start feature to simplify startup and ensure high availability of execution queues in an OpenVMS Cluster. If the node on which the autostart queue is running leaves the OpenVMS Cluster, the queue automatically fails over to the next available node on which autostart is enabled. Autostart is particularly useful on LAT queues. Because LAT printers are usually shared among users of multiple systems or in OpenVMS Cluster systems, many users are affected if a LAT queue is unavailable.
Format for creating autostart queues:
INITIALIZE/QUEUE/AUTOSTART_ON=(node-name::device:,node-name::device:, . . . queue-name
When you use the /AUTOSTART_ON qualifier, you must initially activate the queue for autostart, either by specifying the /START qualifier with the INITIALIZE/QUEUE command or by entering a START/QUEUE command. However, the queue cannot begin processing jobs until the ENABLE AUTOSTART/QUEUES command is entered for a node on which the queue can run. Generic queues cannot be autostart queues.
Rules: Generic queues cannot be autostart queues. Note that you cannot specify both /ON and /AUTOSTART_ON.
Reference: Refer to Section 7.13, “Disabling Autostart During Shutdown” for information about setting the time at which autostart is disabled.
7.7.4. Examples
$ INITIALIZE/QUEUE/ON=JUPITR::LPA0/START/NAME_OF_MANAGER=PRINT_MANAGER JUPITR_LPA0 $ INITIALIZE/QUEUE/ON=JUPITR::LPB0/START/NAME_OF_MANAGER=PRINT_MANAGER JUPITR_LPB0

7.8. Setting Up Clusterwide Generic Print Queues
The clusterwide queue database enables you to establish generic queues that function throughout the cluster. Jobs queued to clusterwide generic queues are placed in any assigned print queue that is available, regardless of its location in the cluster. However, the file queued for printing must be accessible to the computer to which the printer is connected.
7.8.1. Sample Configuration
Figure 7.3, “Clusterwide Generic Print Queue Configuration” illustrates a clusterwide generic print queue in which the queues for all LPA0 printers in the cluster are assigned to a clusterwide generic queue named SYS$PRINT.
A clusterwide generic print queue needs to be initialized and started only once. The most efficient way to start your queues is to create a common command procedure that is executed by each OpenVMS Cluster computer (see Section 7.12.3, “Example”).

7.8.2. Command Example
$ INITIALIZE/QUEUE/GENERIC=(JUPITR_LPA0,SATURN_LPA0,URANUS_LPA0)/START SYS$PRINT
Jobs queued to SYS$PRINT are placed in whichever assigned print queue is available. Thus, in this example, a print job from JUPITR that is queued to SYS$PRINT can be queued to JUPITR_LPA0, SATURN_LPA0, or URANUS_LPA0.
7.9. Setting Up Execution Batch Queues
Generally, you set up execution batch queues on each OpenVMS Cluster computer using the same procedures you use for a standalone computer. For more detailed information about how to do this, see the VSI OpenVMS System Manager's Manual.
7.9.1. Before You Begin
Determine what type of processing will be performed on each computer.
Set up local batch queues that conform to these processing needs.
Decide whether to set up any clusterwide generic queues that will distribute batch job processing across the cluster.
Decide whether to use autostart queues for startup simplicity.
Once you determine the strategy that best suits your needs, you can create a command procedure to set up your queues. Figure 7.4, “Sample Batch Queue Configuration” shows a batch queue configuration for a cluster consisting of computers JUPITR, SATURN, and URANUS.

7.9.2. Batch Command Format
$ INITIALIZE/QUEUE/BATCH/ON=node::[/START][/NAME_OF_MANAGER=
name-of-manager] queue-name
Qualifier |
Description |
---|---|
/ON |
Specifies the computer on which the batch queue runs. |
/START |
Starts the queue. |
/NAME_OF_MANAGER |
Specifies the name of the queue manager if you are running multiple queue managers. |
7.9.3. Autostart Command Format
INITIALIZE/QUEUE/BATCH/AUTOSTART_ON=node::queue-name
When you use the /AUTOSTART_ON qualifier, you must initially activate the queue for autostart, either by specifying the /START qualifier with the INITIALIZE/QUEUE command or by entering a START/QUEUE command. However, the queue cannot begin processing jobs until the ENABLE AUTOSTART/QUEUES command is entered for a node on which the queue can run.
Rule: Generic queues cannot be autostart queues. Note that you cannot specify both /ON and /AUTOSTART_ON.
7.9.4. Examples
$ INITIALIZE/QUEUE/BATCH/ON=JUPITR::/START/NAME_OF_MANAGER=BATCH_QUEUE JUPITR_BATCH $ INITIALIZE/QUEUE/BATCH/ON=SATURN::/START/NAME_OF_MANAGER=BATCH_QUEUE SATURN_BATCH $ INITIALIZE/QUEUE/BATCH/ON=URANUS::/START/NAME_OF_MANAGER=BATCH_QUEUE URANUS_BATCH
Because batch jobs on each OpenVMS Cluster computer are queued to SYS$BATCH by default, you should consider defining a logical name to establish this queue as a clusterwide generic batch queue that distributes batch job processing throughout the cluster (see Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”). Note, however, that you should do this only if you have a common-environment cluster.
7.10. Setting Up Clusterwide Generic Batch Queues
In an OpenVMS Cluster system, you can distribute batch processing among computers to balance the use of processing resources. You can achieve this workload distribution by assigning local batch queues to one or more clusterwide generic batch queues. These generic batch queues control batch processing across the cluster by placing batch jobs in assigned batch queues that are available. You can create a clusterwide generic batch queue as shown in Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”.
A clusterwide generic batch queue needs to be initialized and started only once. The most efficient way to perform these operations is to create a common command procedure that is executed by each OpenVMS Cluster computer (see Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”).
7.10.1. Sample Configuration
In Figure 7.5, “Clusterwide Generic Batch Queue Configuration”, batch queues from each OpenVMS Cluster computer are assigned to a clusterwide generic batch queue named SYS$BATCH. Users can submit a job to a specific queue (for example, JUPITR_BATCH or SATURN_BATCH), or, if they have no special preference, they can submit it by default to the clusterwide generic queue SYS$BATCH. The generic queue in turn places the job in an available assigned queue in the cluster.
If more than one assigned queue is available, the operating system selects the queue that minimizes the ratio (executing jobs/job limit) for all assigned queues.

7.11. Starting Local Batch Queues
Normally, you use local batch execution queues during startup to run batch jobs to start layered products. For this reason, these queues must be started before the ENABLE AUTOSTART command is executed, as shown in the command procedure in Example 7.1, “Sample Commands for Creating OpenVMS Cluster Queues”.
7.11.1. Startup Command Procedure
$ SUBMIT/PRIORITY=255/NOIDENT/NOLOG/QUEUE=node_BATCH LAYERED_PRODUCT.COM $ START/QUEUE node_BATCH $ DEFINE/SYSTEM/EXECUTIVE SYS$BATCH node_BATCH
Submitting the startup command procedure LAYERED_PRODUCT.COM as a high-priority batch job before the queue starts ensures that the job is executed immediately, regardless of the job limit on the queue. If the queue is started before the command procedure was submitted, the queue might reach its job limit by scheduling user batch jobs, and the startup job would have to wait.
7.12. Using a Common Command Procedure
Once you have created queues, you must start them to begin processing batch and print jobs. In addition, you must make sure the queues are started each time the system reboots, by enabling autostart for autostart queues or by entering START/QUEUE commands for nonautostart queues. To do so, create a command procedure containing the necessary commands.
7.12.1. Command Procedure
You can create a common command procedure named, for example, QSTARTUP.COM, and store it on a shared disk. With this method, each node can share the same copy of the common QSTARTUP.COM procedure. Each node invokes the common QSTARTUP.COM procedure from the common version of SYSTARTUP. You can also include the commands to start queues in the common SYSTARTUP file instead of in a separate QSTARTUP.COM file.
7.12.2. Examples
$$ DEFINE/FORM LN_FORM 10 /WIDTH=80 /STOCK=DEFAULT /TRUNCATE $ DEFINE/CHARACTERISTIC 2ND_FLOOR 2 . . .
$ INITIALIZE/QUEUE/AUTOSTART_ON=(JUPITR::LPA0:)/START JUPITR_PRINT $ INITIALIZE/QUEUE/AUTOSTART_ON=(SATURN::LPA0:)/START SATURN_PRINT $ INITIALIZE/QUEUE/AUTOSTART_ON=(URANUS::LPA0:)/START URANUS_PRINT . . .
$ INITIALIZE/QUEUE/BATCH/START/ON=JUPITR:: JUPITR_BATCH $ INITIALIZE/QUEUE/BATCH/START/ON=SATURN:: SATURN_BATCH $ INITIALIZE/QUEUE/BATCH/START/ON=URANUS:: URANUS_BATCH . . .
$ INITIALIZE/QUEUE/START - _$ /AUTOSTART_ON=(JUPITR::LTA1:,SATURN::LTA1,URANUS::LTA1) - _$ /PROCESSOR=LATSYM /FORM_MOUNTED=LN_FORM - _$ /RETAIN=ERROR /DEFAULT=(NOBURST,FLAG=ONE,NOTRAILER) - _$ /RECORD_BLOCKING LN03$PRINT $ $ INITIALIZE/QUEUE/START - _$ /AUTOSTART_ON=(JUPITR::LTA2:,SATURN::LTA2,URANUS::LTA2) - _$ /PROCESSOR=LATSYM /RETAIN=ERROR - _$ /DEFAULT=(NOBURST,FLAG=ONE,NOTRAILER) /RECORD_BLOCKING - _$ /CHARACTERISTIC=2ND_FLOOR LA210$PRINT $
$ ENABLE AUTOSTART/QUEUES/ON=SATURN $ ENABLE AUTOSTART/QUEUES/ON=JUPITR $ ENABLE AUTOSTART/QUEUES/ON=URANUS
$ INITIALIZE/QUEUE/START SYS$PRINT - _$ /GENERIC=(JUPITR_PRINT,SATURN_PRINT,URANUS_PRINT) $
$ INITIALIZE/QUEUE/BATCH/START SYS$BATCH - _$ /GENERIC=(JUPITR_BATCH,SATURN_BATCH,URANUS_BATCH) $
Define all printer forms and characteristics. | |
Initialize local print queues. In the example, these queues are autostart queues and are started automatically when the node executes the ENABLE AUTOSTART/QUEUES command. Although the /START qualifier is specified to activate the autostart queues, they do not begin processing jobs until autostart is enabled. To enable autostart each time the system reboots, add the ENABLE AUTOSTART/QUEUES command to your queue startup command procedure, as shown in Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”. | |
Initialize and start local batch queues on all nodes, including satellite nodes. In this example, the local batch queues are not autostart queues. | |
Initialize queues for remote LAT printers. In the example, these queues are autostart queues and are set up to run on one of three nodes. The queues are started on the first of those three nodes to execute the ENABLE AUTOSTART command. You must establish the logical devices LTA1 and LTA2 in the LAT startup command procedure LAT$SYSTARTUP.COM on each node on which the autostart queue can run. For more information, see the description of editing LAT$SYSTARTUP.COM in the VSI OpenVMS System Manager's Manual. Although the /START qualifier is specified to activate these autostart queues, they will not begin processing jobs until autostart is enabled. |
Enable autostart to start the autostart queues automatically. In the example, autostart is enabled on node SATURN first, so the queue manager starts the autostart queues that are set up to run on one of several nodes. |
Initialize and start the generic output queue SYS$PRINT. This is a nonautostart queue (generic queues cannot be autostart queues). However, generic queues are not stopped automatically when a system is shut down, so you do not need to restart the queue each time a node reboots. |
7.12.3. Example
$! $! QSTARTUP.COM -- Common procedure to set up cluster queues $! $!$ NODE = F$GETSYI("NODENAME") $! $! Determine the node-specific subroutine $! $ IF (NODE .NES. "JUPITR") .AND. (NODE .NES. "SATURN") .AND. (NODE .NES. - "URANUS") $ THEN $ GOSUB SATELLITE_STARTUP $ ELSE
$! $! Configure remote LAT devices. $! $ SET TERMINAL LTA1: /PERM /DEVICE=LN03 /WIDTH=255 /PAGE=60 - /LOWERCASE /NOBROAD $ SET TERMINAL LTA2: /PERM /DEVICE=LA210 /WIDTH=255 /PAGE=66 - /NOBROAD $ SET DEVICE LTA1: /SPOOLED=(LN03$PRINT,SYS$SYSDEVICE:) $ SET DEVICE LTA2: /SPOOLED=(LA210$PRINT,SYS$SYSDEVICE:)
$ START/QUEUE/BATCH 'NODE'_BATCH $ GOSUB 'NODE'_STARTUP $ ENDIF $ GOTO ENDING $! $! Node-specific subroutines start here $!
$ SATELLITE_STARTUP: $! $! Start a batch queue for satellites. $! $ START/QUEUE/BATCH 'NODE'_BATCH $ RETURN $!
$JUPITR_STARTUP: $! $! Node-specific startup for JUPITR:: $! Setup local devices and start nonautostart queues here $! $ SET PRINTER/PAGE=66 LPA0: $ RETURN $! $SATURN_STARTUP: $! $! Node-specific startup for SATURN:: $! Setup local devices and start nonautostart queues here $! . . . $ RETURN $! $URANUS_STARTUP: $! $! Node-specific startup for URANUS:: $! Setup local devices and start nonautostart queues here $! . . . $ RETURN $! $ENDING:
$! Enable autostart to start all autostart queues $! $ ENABLE AUTOSTART/QUEUES $ EXIT
On all large nodes, set up remote devices connected by the LAT. The queues for these devices are autostart queues and are started automatically when the ENABLE AUTOSTART/QUEUES command is executed at the end of this procedure. In the example, these autostart queues were set up to run on one of three nodes. The queues start when the first of those nodes executes the ENABLEAUTOSTART/QUEUES command. The queue remains running as long as one of those nodes is running and has autostart enabled. |
On large nodes, start the local batch queue. In the example, the local batch queues are nonautostart queues and must be started explicitly with START/QUEUE commands. |
On satellite nodes, start the local batch queue. | |
Each node executes its own subroutine. On node JUPITR, set up the line printer device LPA0:. The queue for this device is an autostart queue and is started automatically when the ENABLE AUTOSTART/QUEUES command is executed. |
7.13. Disabling Autostart During Shutdown
By default, the shutdown procedure disables autostart at the beginning of the shutdown sequence. Autostart is disabled to allow autostart queues with failover lists to fail over to another node. Autostart also prevents any autostart queue running on another node in the cluster to fail over to the node being shutdown.
7.13.1. Options
Option |
Description |
---|---|
1 |
Define the logical name SHUTDOWN$DISABLE_AUTOSTART as
follows:
$ DEFINE/SYSTEM/EXECUTIVE SHUTDOWN$DISABLE_AUTOSTART number-of-minutes Set the value of |
2 |
Specify the DISABLE_AUTOSTART
|
Reference: See the VSI OpenVMS System Manager's Manual for more information about changing the time at which autostart is disabled during the shutdown sequence.
Chapter 8. Configuring an OpenVMS Cluster System
This chapter provides an overview of the cluster configuration command procedures and describes the preconfiguration tasks required before running either command procedure. Then it describes each major function of the command procedures and the post-configuration tasks, including running AUTOGEN.COM.
8.1. Overview of the Cluster Configuration Procedures
Two similar command procedures are provided for configuring and reconfiguring an OpenVMS Cluster system: CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. The choice depends on whether you use the LANCP utility or DECnet for satellite booting in your cluster. CLUSTER_CONFIG_LAN.COM provides satellite booting services with the LANCP utility; CLUSTER_CONFIG.COM provides satellite booting services with DECnet.
Also, to configure an Integrity server system use CLUSTER_CONFIG_LAN.COM and to configure an Alpha system, use either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM. You can use only CLUSTER_CONFIG_LAN.COM for configuring Cluster over IP.
In a satellite environment, you may want to determine which command procedure is used for configuring a cluster. To determine whether CLUSTER_CONFIG or CLUSTER_CONFIG_LAN is used in cluster configuration, see the SYS$SYSTEM:MODPARAMS.DAT file. While configuring a cluster, the command procedure name is added as a comment in the MODPARAMS.DAT file.
! CLUSTER_CONFIG creating for ADD operation on 4-APR-2009 14:21:00.89
! CLUSTER_CONFIG_LAN creating for ADD operation on 5-APR-2009 14:21:00.89
Similar comments are added for the 'CHANGE' operation. For multiple entries in MODPARAMS.DAT, the last entry reflects the latest procedure name that is used to configure the cluster. See Section 4.5, “Configuring and Starting a Satellite Booting Service” for the factors to consider when choosing a satellite booting service.
Add a computer to the cluster
Remove a computer from the cluster
Change a computer's characteristics
Create a duplicate system disk
Make a directory structure for a new root on a system disk
Delete a root from a system disk
By selecting the appropriate option, you can configure the cluster easily and reliably without invoking any OpenVMS utilities directly. Table 8.1, “Summary of Cluster Configuration Functions ” summarizes the functions that the configuration procedures perform for each configuration option.
The phrase cluster configuration command procedure, when used in this chapter, refers to both CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. The questions of the two configuration procedures are identical except where they pertain to LANCP and DECnet.
Option | Functions Performed |
---|---|
ADD |
Enables a node as a cluster member:
|
REMOVE |
Disables a node as a cluster member:
|
CHANGE |
Displays the CHANGE menu and prompts for appropriate
information to:
|
CREATE |
Duplicates the local computer's system disk and removes all system roots from the new disk. |
MAKE |
Creates a directory structure for a new root on a system disk. |
DELETE |
Deletes a root from a system disk. |
8.1.1. Before Configuring the System
Task | Procedure |
---|---|
Determine whether the computer uses DECdtm. |
When you add a computer to or remove a computer from a cluster that uses DECdtm services, there are a number of tasks you need to do in order to ensure the integrity of your data. Reference: See the chapter about DECdtm services in the VSI OpenVMS System Manager's Manual for step-by-step instructions on setting up DECdtm in an OpenVMS Cluster system. If you are not sure whether your cluster uses DECdtm
services, enter this command
sequence:
$ SET PROCESS /PRIVILEGES=SYSPRV $ RUN SYS$SYSTEM:LMCP LMCP> SHOW LOG If your cluster does not use DECdtm services, the SHOW LOG command will display a “file not found” error message. If your cluster uses DECdtm services, it displays a list of the files that DECdtm uses to store information about transactions. |
Ensure the network software providing the satellite booting service is up and running and all computers are connected to the LAN. |
For nodes that will use the LANCP utility for satellite
booting, run the LANCP utility and enter the LANCP command
LIST DEVICE/MOPDLL to display a list of LAN devices on the
system:
$ RUN SYS$SYSTEM:LANCP LANCP> LIST DEVICE/MOPDLL For nodes running DECnet for OpenVMS, enter the DCL
command SHOW NETWORK to determine whether the network is up
and
running:
$ SHOW NETWORK Product: DECNET Node: CHBOSE Address(es): 25.169 Product: TCP/IP Node: chbose.ind.hp.com Address(es): 18.156.235.23 This example shows that the node CHBOSE is running DECnet for OpenVMS and node chbose.ind.hp.com is running TCP/IP. If DECnet has not been started, the message “SHOW-I-NONET, Network Unavailable” is displayed. For nodes running DECnet–Plus, refer to VSI OpenVMS DECnet Network Management Utilities for information about determining whether the DECnet–Plus network is up and running. |
Select MOP and disk servers. |
Every OpenVMS Cluster configured with satellite nodes must include at least one Maintenance Operations Protocol (MOP) and disk server. When possible, select multiple computers as MOP and disk servers. Multiple servers give better availability, and they distribute the work load across more LAN adapters. Follow these guidelines when selecting MOP and disk servers:
|
Make sure you are logged in to a privileged account. |
Log in to a privileged account. Rules: If you are adding a satellite, you must be logged into the system manager's account on a boot server. Note that the process privileges SYSPRV, OPER, CMKRNL, BYPASS, and NETMBX are required, because the procedure performs privileged system operations. |
Coordinate cluster common files. |
If your configuration has two or more system disks, follow the instructions in Chapter 5, Preparing a Shared Environment to coordinate the cluster common files. |
Optionally, disable broadcast messages to your terminal. |
While adding and removing computers, many such messages are generated. To disable the messages, you can enter the DCL command REPLY/DISABLE=(NETWORK, CLUSTER). See also Section 10.5, “Controlling OPCOM Messages” for more information about controlling OPCOM messages. |
Predetermine answers to the questions asked by the cluster configuration procedure. |
Table 8.3, “Data Requested by CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM” describes the data requested by the cluster configuration command procedures. |
8.1.2. Data Requested by the Cluster Configuration Procedures
The following table describes the questions asked by the cluster configuration command procedures and describes how you might answer them. The table is supplied here so that you can determine answers to the questions before you invoke the procedure.
Information Required | How to Specify or Obtain |
---|---|
For all configurations | |
Device name of cluster system disk on which root directories will be created |
Press Return to accept the default device name which is the translation of the SYS$SYSDEVICE: logical name, or specify a logical name that points to the common system disk. |
Computer's root directory name on cluster system disk |
Press Return to accept the
procedure-supplied default, or specify a name in the form
SYS
x :
|
Workstation windowing system |
System manager specifies. Workstation software must be installed before workstation satellites are added. If it is not, the procedure indicates that fact. |
Location and sizes of page and swap files |
This information is requested only when you add a computer to the cluster. Press Return to accept the default size and location. (The default sizes displayed in brackets by the procedure are minimum values. The default location is the device name of the cluster system disk). If your configuration includes satellite nodes, you may realize a performance improvement by locating satellite page and swap files on a satellite's local disk, if such a disk is available. The potential for performance improvement depends on the configuration of your OpenVMS Cluster system disk and network. To set up page and swap files on a satellite's local disk,
the cluster configuration procedure creates a command
procedure called SATELLITE_PAGE.COM in the satellite's [SYS
n.SYSEXE] directory on the boot
server's system disk. The SATELLITE_PAGE.COM procedure
performs the following functions:
Note: For page and swap disks that are shadowed, you must edit the MOUNT and INIT commands in SATELLITE_PAGE.COM to the appropriate syntax for mounting any specialized “local” disks (that is, host-based shadowing disks (DS xxx), or host-based RAID disks (DP xxxx), or DECram disks (MDA xxxx)) on the newly added node. CLUSTER_CONFIG(_LAN).COM does not create the MOUNT and INIT commands required for SHADOW, RAID, or DECram disks.
Note: To relocate the
satellite's page and swap files (for example, from the
satellite's local disk to the boot server's system disk, or
the reverse) or to change file sizes:
|
Value for local computer's allocation class (ALLOCLASS or TAPE_ALLOCLASS) parameter. |
The ALLOCLASS parameter can be used for a node allocation class or, on Alpha computers, a port allocation class. Refer to Section 6.2.1, “Allocation Classes” for complete information about specifying allocation classes. |
Physical device name of quorum disk |
System manager specifies. |
For systems running DECnet for OpenVMS | |
Computer's DECnet node address for Phase IV |
For the DECnet node address, you obtain this information
as follows:
|
Computer's DECnet node name |
Network manager supplies. The name must be from 1 to 6 alphanumeric characters and cannot include dollar signs ($) or underscores (_). |
For systems running DECnet–Plus | |
Computer's DECnet node address for Phase IV(if you need Phase IV compatibility) |
For the DECnet node address, you obtain this information
as follows:
|
Node's DECnet full name |
Determine the full name with the help of your network
manager. Enter a string comprised of:
|
SCS node name for this node |
Enter the OpenVMS Cluster node name, which is a string of 6or fewer alphanumeric characters. |
DECnet synonym |
Press Return to define a DECnet synonym, which is a short name for the node's full name. Otherwise, enter N. |
Synonym name for this node |
Enter a string of 6 or fewer alphanumeric characters. By
default, it is the first 6 characters of the last simple
name in the full name. For example:
Note: The node synonym does not need to be the same as the OpenVMS Cluster node name. |
MOP service client name for this node |
Enter the name for the node's MOP service client when the node is configured as a boot server. By default, it is the OpenVMS Cluster node name (for example, the SCS node name). This name does not need to be the same as the OpenVMS Cluster node name. |
For systems running TCP/IP or the LANCP Utility for satellite booting, or both | |
Computer's SCS node name (SCSNODE) and SCS system ID (SCSSYSTEMID) |
These prompts are described in Section 4.2.3, “Information Required”. If a system is running TCP/IP, the procedure does not ask for a TCP/IP host name because a cluster node name (SCSNODE) does not have to match a TCP/IP host name. The TCP/IP host name might be longer than six characters, whereas the SCSNODE name must be no more than six characters. Note that if the system is running both DECnet and IP, then the procedure uses the DECnet defaults. |
For LAN configurations | |
Cluster group number and password |
This information is requested only when the CHANGE option is chosen. See Section 2.5, “OpenVMS Cluster Membership” for information about assigning cluster group numbers and passwords. |
Satellite's LAN hardware address |
Address has the form On Integrity
servers:
Shell> lanaddress On Alpha systems:
>>> SHOW NETWORK These commands display the hardware address of the LAN devices that can be used for satellite booting. Note that you can also use the SHOW CONFIG command at LANCP. |
For IP configurations | |
UDP port number |
UDP port number is the port number used for cluster communication. UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number or this port number is used by any other application. |
IP multicast address |
Enter the IP multicast address for cluster, if IP
multicasting is enabled. By default, the IP multicast
address is selected from the administratively scoped IP
multicast address range of 239.242.x.y. The last two octets
x and y are generated based on the cluster group number. For
example, if the cluster group number is 1985, the multicast
is calculated as
follows:
X= 1985/256 Y= 1985 - (256 *x) The system administrator can override the default multicast address with a unique address for their environment. |
IP unicast address |
If the node that is configured uses IP unicast to discover a remote note, you need the IP unicast address of the existing members or any new member in the cluster. |
IP address |
It is the IP address of the local system from where the cluster is configured. |
Gateway and Network mask address |
In the configuration option, select option 4 to add the TCP/IP gateway and network mask address to the cluster over IP database. |
IP interface address |
In the configuration option for the selected address, select option 4 to add to the cluster over IP database. The interface information along with the default route is entered in the TCPIP$CLUSTER.DAT as shown in the following example: interface=IE0,EIA0,10.0.1.2,255.255.255.0 default_route=10.0.1.1 |
IP interface address for satellite booting |
To select the IP interface to be used for satellite booting. For Alpha systems:
|
8.1.3. Invoking the Procedure
$ @CLUSTER_CONFIG_LAN
$ @CLUSTER_CONFIG
Caution: Do not invoke multiple sessions simultaneously. You can run only one cluster configuration session at a time.
Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet Phase IV is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". CALVIN is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for CALVIN. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: . . . . .
This chapter contains a number of sample sessions showing how to run the cluster configuration procedures. Although the CLUSTER_CONFIG_LAN.COM and the CLUSTER_CONFIG.COM procedure function the same for Integrity server systems and Alpha systems, the questions and format may appear slightly different according to the type of computer system.
8.2. Adding Computers
IF... | THEN... |
---|---|
You are adding your first satellite node to the OpenVMS Cluster. |
Follow these steps:
|
The cluster uses DECdtm services. |
You must create a transaction log for the computer when you have configured it into your cluster. For step-by-step instructions on how to do this, see the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual. |
You add a CI connected computer that boots from a cluster common system disk. |
You must create a new default bootstrap command procedure for the computer before booting it into the cluster. For instructions, refer to your computer-specific installation and operations guide. |
You are adding computers to a cluster with more than one common system disk. |
You must use a different device name for each system disk on which computers are added. For this reason, the cluster configuration procedure supplies as a default device name the logical volume name (for example, DISK$MARS_SYS1) of SYS$SYSDEVICE: on the local system. Using different device names ensures that each computer added has a unique root directory specification, even if the system disks contain roots with the same name—for example, DISK$MARS_SYS1:[SYS10] and DISK$MARS_SYS2:[SYS10]. |
You add a voting member to the cluster. |
You must, after the ADD function completes, reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”. |
Caution: If either the local or the new computer fails before the ADD function completes, you must, after normal conditions are restored, perform the REMOVE option to erase any invalid data and then restart the ADD option. Section 8.3, “Removing Computers” describes the REMOVE option.
8.2.1. Controlling Conversational Bootstrap Operations
When you add a satellite to the cluster using either cluster configuration command procedure, the procedure asks whether you want to allow conversational bootstrap operations for the satellite (default is No).
If you select the default, the NISCS_CONV_BOOT system parameter in the satellite's system parameter file remains set to 0 to disable such operations. The parameter file (IA64VMSSYS.PAR for Integrity servers or ALPHAVMSSYS.PAR for Alpha systems) resides in the satellite's root directory on a boot server's system disk (device:[SYS x.SYSEXE]). You can enable conversational bootstrap operations for a given satellite at any time by setting this parameter to 1.
Example:
Step |
Action |
---|---|
1 |
Log in as system manager on the boot server. |
2 |
On Integrity servers or Alpha systems, invoke the System
Generation utility (SYSGEN) and enter the following
commands:
$ RUN SYS$SYSTEM:SYSGEN SYSGEN> USE $1$DGA11:[SYS10.SYSEXE]ALPHAVMSSYS.PAR SYSGEN> SET NISCS_CONV_BOOT 1 SYSGEN> WRITE $1$DGA11:[SYS10.SYSEXE]ALPHAVMSSYS.PAR SYSGEN> EXIT $ |
3 |
Modify the satellite's MODPARAMS.DAT file so that NISCS_CONV_BOOT is set to 1. |
8.2.2. Common AUTOGEN Parameter Files
WHEN the node being added is a... |
THEN... |
---|---|
Satellite node |
The following line is added to the MODPARAMS.DAT
file:
AGEN$INCLUDE_PARAMS SYS$MANAGER:AGEN$NEW_SATELLITE_DEFAULTS.DAT |
Nonsatellite node |
The following line is added to the MODPARAMS.DAT
file:
AGEN$INCLUDE_PARAMS SYS$MANAGER:AGEN$NEW_NODE_DEFAULTS.DAT |
The AGEN$NEW_SATELLITE_DEFAULTS.DAT and AGEN$NEW_NODE_DEFAULTS.DAT files hold AUTOGEN parameter settings that are common to all satellite nodes or nonsatellite nodes in the cluster. Use of these files simplifies system management, because you can maintain common system parameters in either one or both of these files. When adding or changing the common parameters, this eliminates the need to make modifications in the MODPARAMS.DAT files located on every node in the cluster.
Initially, these files contain no parameter settings. You edit the AGEN$NEW_SATELLITE_DEFAULTS.DAT and AGEN$NEW_NODE_DEFAULTS.DAT files, as appropriate, to add, modify, or edit system parameters. For example, you might edit the AGEN$NEW_SATELLITE_DEFAULTS.DAT file to set the MIN_GBLPAGECNT parameter to 5000. AUTOGEN makes the MIN_GBLPAGECNT parameter and all other parameter settings in the AGEN$NEW_SATELLITE_DEFAULTS.DAT file common to all satellite nodes in the cluster.
AUTOGEN uses the parameter settings in the AGEN$NEW_SATELLITE_DEFAULTS.DAT or AGEN$NEW_NODE_DEFAULTS.DAT files the first time it is run, and with every subsequent execution.
8.2.3. Examples
Examples 8.1, 8.2, and 8.3 describes the use of CLUSTER_CONFIG_LAN.COM on BHAGAT to add, respectively, a boot server running DECnet for OpenVMS, a boot server running DECnet–Plus, and a satellite node.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 1 This ADD function will add a new IA64 node to the cluster. WARNING: If the node being added is a voting member, EXPECTED_VOTES for every cluster member must be adjusted. For complete instructions check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. CAUTION: If this cluster is running with multiple system disks and common system files will be used, please, do not proceed unless appropriate logical names are defined for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. If this cluster will run IPCI, then TCP/IP installed on the system should be version 5.7 and above or else IPCI configuration will be aborted. Do you want to continue [Y]?[Return] Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y Will the node be a satellite [Y]? N What is the node's SCS node name? MOON What is the node's SCSSYSTEMID number? 24.123 NOTE: 24.123 equates to an SCSSYSTEMID of 24699 Will MOON be a boot server [Y]? [Return] TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes. Use SYS$MANAGER:TCPIP$CONFIG.COM on MOON to enable BOOTP and TFTP service after MOON has booted into the cluster. This procedure will now ask you for the device name of MOON's system root. The default device name (DISK$BHAGAT_831H1:) is the logical volume name of SYS$SYSDEVICE:. What is the device name for MOON's system root [default DISK$BHAGAT_831H1:]? What is the name of MOON's system root [SYS1]? [Return] Creating directory tree SYS1 ... System root SYS1 created ENABLE IP for cluster communications (Y/N)? N CAUTION: If you do not define port allocation classes later in this procedure for shared SCSI buses, all nodes sharing a SCSI bus must have the same non-zero ALLOCLASS value. If multiple nodes connect to a shared SCSI bus without the same allocation class for the bus, system booting will halt due to the error or IO AUTOCONFIGURE after boot will keep the bus offline. WARNING: If BHAGAT is sharing the same SCSI bus with MOON, then BHAGAT's ALLOCLASS parameter or port allocation class for the shared bus must be changed from 0 to the same non-zero value that will be entered for MOON. Use the CHANGE option of CLUSTER_CONFIG_LAN.COM to change BHAGAT's ALLOCLASS parameter before MOON is booted. Enter a value for MOON's ALLOCLASS parameter [1]: [Return] Does this cluster contain a quorum disk [N]? [Return] Size of pagefile for MOON [RETURN for AUTOGEN sizing]? [Return] A temporary pagefile will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary pagefile [10000]? [Return] Size of swap file for MOON [RETURN for AUTOGEN sizing]? [Return] A temporary swap file will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary swap file [8000]? [Return] Each shared SCSI bus must have a positive allocation class value. A shared bus uses a PK adapter. A private bus may use: PK, DR, DV, DQ. When adding a node with SCSI-based cluster communications, the shared SCSI port allocation classes may be established in SYS$DEVICES.DAT. Otherwise, the system's disk allocation class will apply. A private SCSI bus need not have an entry in SYS$DEVICES.DAT. If it has an entry, its entry may assign any legitimate port allocation class value: n where n = a positive integer, 1 to 32767 inclusive 0 no port allocation class and disk allocation class does not apply -1 system's disk allocation class applies (system parameter ALLOCLASS) When modifying port allocation classes, SYS$DEVICES.DAT must be updated for all affected nodes, and then all affected nodes must be rebooted. The following dialog will update SYS$DEVICES.DAT on MOON. Enter [RETURN] to continue: There are currently no entries in SYS$DEVICES.DAT for MOON. After the next boot, any SCSI controller on MOON will use MOON's disk allocation class. Assign port allocation class to which adapter [RETURN for none]: [Return] Will a disk local only to MOON (and not accessible at this time to BHAGAT) be used for paging and swapping (Y/N)? N If you specify a device other than DISK$BHAGAT_831H1: for MOON's page and swap files, this procedure will create PAGEFILE_MOON.SYS and SWAPFILE_MOON.SYS in the <SYSEXE> directory on the device you specify. What is the device name for the page and swap files [DISK$BHAGAT_831H1:]? %SYSGEN-I-CREATED, BHAGAT$DKA100:<SYS1.SYSEXE>PAGEFILE.SYS;1 created %SYSGEN-I-CREATED, BHAGAT$DKA100:<SYS1.SYSEXE>SWAPFILE.SYS;1 created The configuration procedure has completed successfully. MOON has been configured to join the cluster. The first time MOON boots, AUTOGEN.COM will run automatically.
$ @CLUSTER_CONFIG.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an Alpha System DECnet-Plus is installed on this node. Alpha satellites will use LANCP, not DECnet, for MOP downline loading. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BISMIL is an Alpha system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an Alpha node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BISMIL. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 1 This ADD function will add a new Alpha node to the cluster. WARNING: If the node being added is a voting member, EXPECTED_VOTES for every cluster member must be adjusted. For complete instructions check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. CAUTION: If this cluster is running with multiple system disks and common system files will be used, please, do not proceed unless appropriate logical names are defined for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. If this cluster will run IPCI, then TCP/IP installed on the system should be version 5.7 and above or else IPCI configuration will be aborted. Do you want to continue [Y]? [Return] Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y Will the node be a satellite [Y]? N What is the node's SCS node name? MOON DECnet is running on this node. Even though you are configuring a LAN- based cluster, the DECnet database will provide some information and may be updated. What is the node's DECnet full name? local:.MOON Do you want to define a DECnet synonym [Y]? N What is the MOP service client name for this node [MOON]? VENUS What is the node's SCSSYSTEMID number? 24.123 NOTE: 24.123 equates to an SCSSYSTEMID of 24699 Will MOON run DECnet [Y]? [Return] Note: This procedure will not update any network databases with information about MOON. You must do that yourself. Will MOON be a boot server [Y]? [Return] This procedure will now ask you for the device name of MOON's system root. The default device name (DISK$ALPHA732:) is the logical volume name of SYS$SYSDEVICE:. What is the device name for MOON's system root [default DISK$ALPHA732:]? What is the name of MOON's system root [SYS1]? [Return] Creating directory tree SYS1 ... System root SYS1 created ENABLE IP for cluster communications (Y/N)? N CAUTION: If you do not define port allocation classes later in this procedure for shared SCSI buses, all nodes sharing a SCSI bus must have the same non-zero ALLOCLASS value. If multiple nodes connect to a shared SCSI bus without the same allocation class for the bus, system booting will halt due to the error or IO AUTOCONFIGURE after boot will keep the bus offline. WARNING: If BISMIL is sharing the same SCSI bus with MOON, then BISMIL's ALLOCLASS parameter or port allocation class for the shared bus must be changed from 0 to the same non-zero value that will be entered for MOON. Use the CHANGE option of CLUSTER_CONFIG_LAN.COM to change BISMIL's ALLOCLASS parameter before MOON is booted. Enter a value for MOON's ALLOCLASS parameter [1]: [Return] Does this cluster contain a quorum disk [N]? [Return] Size of pagefile for MOON [RETURN for AUTOGEN sizing]? [Return] A temporary pagefile will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary pagefile [10000]? [Return] Size of swap file for MOON [RETURN for AUTOGEN sizing]? [Return] A temporary swap file will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary swap file [8000]? [Return] Each shared SCSI bus must have a positive allocation class value. A shared bus uses a PK adapter. A private bus may use: PK, DR, DV, DQ. When adding a node with SCSI-based cluster communications, the shared SCSI port allocation classes may be established in SYS$DEVICES.DAT. Otherwise, the system's disk allocation class will apply. A private SCSI bus need not have an entry in SYS$DEVICES.DAT. If it has an entry, its entry may assign any legitimate port allocation class value: n where n = a positive integer, 1 to 32767 inclusive 0 no port allocation class and disk allocation class does not apply -1 system's disk allocation class applies (system parameter ALLOCLASS) When modifying port allocation classes, SYS$DEVICES.DAT must be updated for all affected nodes, and then all affected nodes must be rebooted. The following dialog will update SYS$DEVICES.DAT on MOON. Enter [RETURN] to continue: [Return] There are currently no entries in SYS$DEVICES.DAT for MOON. After the next boot, any SCSI controller on MOON will use MOON's disk allocation class. Assign port allocation class to which adapter [RETURN for none]: Will a local (non-HSx) disk on MOON and not on a hierarchical storage controller be used for paging and swapping (Y/N)? N If you specify a device other than DISK$ALPHA732: for MOON's page and swap files, this procedure will create PAGEFILE_MOON.SYS and SWAPFILE_MOON.SYS in the <SYSEXE> directory on the device you specify. What is the device name for the page and swap files [DISK$ALPHA732:]? %SYSGEN-I-CREATED, BISMIL$DKB100:<SYS1.SYSEXE>PAGEFILE.SYS;1 created %SYSGEN-I-CREATED, BISMIL$DKB100:<SYS1.SYSEXE>SWAPFILE.SYS;1 created The configuration procedure has completed successfully. MOON has been configured to join the cluster. Before booting MOON, you must create a new default bootstrap command procedure for MOON. For instructions, see your processor-specific installation and operations guide. The first time MOON boots, NET$CONFIGURE.COM and AUTOGEN.COM will run automatically. The following parameters have been set for MOON: VOTES = 1 QDSKVOTES = 1 After MOON has booted into the cluster, you must increment the value for EXPECTED_VOTES in every cluster member's MODPARAMS.DAT. You must then reconfigure the cluster, using the procedure described in the "OpenVMS Cluster Systems" manual.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 1 This ADD function will add a new IA64 node to the cluster. WARNING: If the node being added is a voting member, EXPECTED_VOTES for every cluster member must be adjusted. For complete instructions check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. CAUTION: If this cluster is running with multiple system disks and common system files will be used, please, do not proceed unless appropriate logical names are defined for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. If this cluster will run IPCI, then TCP/IP installed on the system should be version 5.7 and above or else IPCI configuration will be aborted. Do you want to continue [Y]? [Return] Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N Will the node be a satellite [Y]? [Return] What is the node's SCS node name? GOMTHI DECnet is running on this node. Even though you are configuring a LAN- based cluster, the DECnet database will provide some information and may be updated. What is the node's DECnet full name? local:.GOMTHI Do you want to define a DECnet synonym [Y]? N What is the node's SCSSYSTEMID number? 25.171 NOTE: 25.171 equates to an SCSSYSTEMID of 25771 WARNING: The DECnet databases on BHAGAT will not be updated with information on GOMTHI. You must see to it that network databases on this and all other cluster members are updated. For help, refer to the "OpenVMS Cluster Systems" manual. Does GOMTHI need to be registered in the DECnet namespace [N]?[Return] What is the Cluster Alias full name? [Return] Will GOMTHI run DECnet [Y]? [Return] This procedure will now ask you for the device name of GOMTHI's system root. The default device name (DISK$BHAGAT_SYS:) is the logical volume name of SYS$SYSDEVICE:. What is the device name for GOMTHI's system root [default DISK$BHAGAT_SYS:]? What is the name of GOMTHI's system root [SYS10]? [Return] What is GOMTHI's LAN adapter hardware address? 00-30-6E-4C-BB-1A What is GOMTHI's TCP/IP address? 16.181.160.129 Would you like GOMTHI added as a TCP/IP host shortcut for 16.181.160.129 [Y]? [Return] What is GOMTHI's TCP/IP gateway or gateways (leave blank if none)? 16.181.160.1 What is GOMTHI's TCP/IP network mask [255.255.255.0]? 255.255.252.0 NOTE: Make sure to set the VMS_FLAGS console variable to 0,200000 on node GOMTHI so it will use the memory-disk method to boot as a satellite. The command to update this variable from the console EFI shell of GOMTHI is: set vms_flags "0,200000" Allow conversational bootstraps on GOMTHI [N]? [Return] The following workstation windowing options are available: 1. No workstation software 2. DECwindows Workstation Software Enter choice [1]: [Return] Creating directory tree SYS10 ... System root SYS10 created ENABLE IP for cluster communications (Y/N)? N Will GOMTHI be a disk server [N]? Y Enter a value for GOMTHI's ALLOCLASS parameter [0]: [Return] Updating BOOTP database with satellite information for GOMTHI... Size of pagefile for GOMTHI [RETURN for AUTOGEN sizing]? [Return] A temporary pagefile will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary pagefile [10000]? [Return] Size of swap file for GOMTHI [RETURN for AUTOGEN sizing]? [Return] A temporary swap file will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary swap file [8000]? [Return] NOTE: IA64 satellite node GOMTHI requires DOSD if capturing the system state in a dumpfile is desired after a system crash. Will a local disk on GOMTHI be used for paging and swapping (Y/N)? Y This procedure will now wait until GOMTHI is a member of the cluster. Once GOMTHI joins the cluster, this procedure will ask you which local disk it can use for paging and swapping. Please boot GOMTHI now. Make sure the default boot device is set to the appropriate clustered-disk access path: LAN device for satellite nodes; or shared-bus (CI/DSSI/SCSI/FC) disk device. See the hardware user manual or the console help command for instructions to do this. Waiting for GOMTHI to boot... Waiting for GOMTHI to boot... Waiting for GOMTHI to boot... Waiting for GOMTHI to boot... Waiting for GOMTHI to boot... Node GOMTHI is now a cluster member. This procedure will pause for up to 4 minutes, while attempting to detect local disks on GOMTHI, to use for paging and swapping. The local disks on GOMTHI are: Device Device Error Volume Free Trans Mnt Name Status Count Label Blocks Count Cnt GOMTHI$DQA0: Online 0 GOMTHI$DKA0: Online 0 GOMTHI$DKA100: Online 0 GOMTHI$DKB200: Online 0 If the paging and swapping disk you plan to use is not displayed, it may not yet be configured. Please wait a few moments and hit a carriage return for an updated display. Which disk can be used for paging and swapping? GOMTHI$DKA100: May this procedure INITIALIZE GOMTHI$DKA100 [Y]? N In order to ensure that this disk has a unique volume name this procedure wishes to change its name from [GOMTHI_831H1] to [GOMTHI_25771]. If the satellite being added may also be booted standalone and refers to this disk by name you may retain the old volume name if there are no other disks with the same name in this cluster. May the volume name of this disk be changed to GOMTHI_25771 [Y]? N %DELETE-W-SEARCHFAIL, error searching for SYS$COMMON:[SYSMGR]CLU2020042F.TMP1;* -RMS-E-FNF, file not found Mounting GOMTHI$DKA100... What is the file specification for the pagefile on GOMTHI$DKA100: [ <SYSEXE>PAGEFILE.SYS ]? [Return] %CREATE-I-CREATED, GOMTHI$DKA100:<SYSEXE> created %SYSGEN-I-CREATED, GOMTHI$DKA100:<SYSEXE>PAGEFILE.SYS;1 created What is the file specification for the swapfile on GOMTHI$DKA100: [ <SYSEXE>SWAPFILE.SYS ]? [Return] %SYSGEN-I-CREATED, GOMTHI$DKA100:<SYSEXE>SWAPFILE.SYS;1 created SATELLITE_PAGE.COM and INSTALL_PAGE.COM will now be created for local page/swap disk/file installation. ****** ! SHADOWED PAGE or SWAP DISK WARNING ! ****** **** Edit these procedures to include any **** **** local configuration commands necessary for **** **** shadowed disk mounting, prior to reboot. **** ***************************************************** AUTOGEN will now reconfigure and reboot GOMTHI automatically. These operations will complete in a few minutes, and a completion message will be displayed at your terminal. Waiting for GOMTHI to reboot... Waiting for GOMTHI to reboot... The configuration procedure has completed successfully.
8.2.3.1. Creating and Configuring a Two-Node Cluster Using IP
Cluster over IP can be used to create and configure a two-node cluster. Node ORCHID is a standalone node at SITE A and node TULIP at SITE B, is already a member (the only member) of the cluster. In this scenario, Cluster over IP is not configured in TULIP. SITE A and SITE B can be in the same or different LAN, building or any other geographical location. It is required to have IP connectivity between SITE A and SITE B and should be within the supported inter site distance.
Step 1. Configuring Node TULIP to Enable Cluster over IP
TULIP$@SYS$MANAGER:CLUSTER_CONFIG_LAN Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an Alpha System DECnet-Plus is installed on this node. Alpha satellites will use LANCP, not DECnet, for MOP downline loading. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". TULIP is an Alpha system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an Alpha node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for TULIP. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3CHANGE Menu 1. Enable TULIP as a boot server. 2. Disable TULIP as a boot server. 3. Enable a quorum disk for TULIP 4. Disable a quorum disk for TULIP. 5. Enable TULIP as a disk server. 6. Disable TULIP as a disk server. 7. Change TULIP's ALLOCLASS value. 8. Enable TULIP as a tape server. 9. Disable TULIP as a tape server. 10. Change TULIP's TAPE_ALLOCLASS value. 11. Change an Alpha satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on TULIP. 13. Disable Cluster Communication using IP on TULIP. 14. Enable the LAN for cluster communications on TULIP. 15. Disable the LAN for cluster communications on TULIP. 16. Enable Memory Channel for cluster communications on TULIP. 17. Disable Memory Channel for cluster communications on TULIP. 18. Change TULIP's shared SCSI port allocation class value. 19. Return to MAIN menu. Enter choice [19]: 12
ENABLE IP for cluster communications (Y/N)? Y
UDP port number to be used for Cluster Communication over IP[49152]?Y
Enable IP multicast for cluster communication(Y/N)[Y]? Y
What is the IP multicast address[239.242.7.193]? [Return]
What is the TTL (time to live) value for IP multicast packets [32]? [Return]
Do you want to enter unicast address(es)(Y/N)[Y]? [Return]
What is the unicast address [Press Enter to end the list]? 10.0.1.2
What is the unicast address[Press Enter to end the list]? [Return]
***************************************************************** Cluster Communications over IP has been enabled. Now CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure. Please select the IP interfaces to be used for Cluster Communications over IP (IPCI). This can be done selecting "Core Environment" option from the main menu followed by the "Interfaces" option. You may also use this opportunity to configure other aspects. **************************************************************** Press Enter to continue. Checking TCP/IP Services for OpenVMS configuration database files. TCP/IP Services for OpenVMS Configuration Menu Configuration options: 1 - Core environment 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: 1
TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: 2
TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=TULIP, Active=TULIP Configuration options: 0 - Set The Target Node (Current Node: TULIP) 1 - IE0 Menu (EIA0: TwistedPair 100mbps) 2 - 10.0.2.2/23 TULIP Configured,Active 3 - IE1 Menu (EIB0: TwistedPair 100mbps) 4 - 10.0.2.224/23 *noname* Configured,Active I - Information about your configuration [E] - Exit menu Enter configuration option: 2
TCP/IP Services for OpenVMS Address Configuration Menu (Node: TULIP) IE0 10.0.2.2/23 TULIP Configured, Active IE0 Configuration options 1 - Change address 2 - Set "TULIP" as the default hostname 3 - Delete from configuration database 4 - Add to IPCI database 5 - Deactivate from live system 6 - Add standby aliases to configuration database (for failSAFE IP) [E] - Exit menu Enter configuration option: 4
Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT; Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT; Added address IE1:10.0.2.2 to IPCI database TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=TULIP, Active=TULIP Configuration options: 0 - Set The Target Node (Current Node:tulip) 1 - IE0 Menu (EIA0: TwistedPair 100mbps) 2 - 10.0.2.2/23 TULIP Configured,IPCI,Active 3 - IE1 Menu (EIB0: TwistedPair 100mbps) 4 - 10.0.2.224/23 *noname* Configured,Active I - Information about your configuration [E]- Exit menu Enter configuration option: E
TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: E TCP/IP Services for OpenVMS Configuration Menu Configuration options: 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: E The configuration procedure has completed successfully. Tulip has been enabled for IP communications
Please run AUTOGEN to reboot TULIP: TULIP$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
TULIP is a single-member cluster without Cluster over IP enabled. The cluster member characteristics can be changed to enable Cluster over IP for this node by selecting option 3. |
Select option 12 to enable cluster over IP. By selecting this option, the SYSGEN parameter, NISCS_USE_UDP is set to 1, which enables the PEDRIVER to use IP for cluster communication. This requires reboot of the node. If LAN is not already selected as the cluster interconnect, this option sets NISCS_LOAD_PEA0 to 1 to load PEDRIVER during the next reboot. |
You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator, if IP multicasting is enabled in your environment. |
You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator, if IP multicasting is enabled in your environment. |
Enter the IP multicast address for the cluster, if IP
multicasting is enabled. By default, the IP multicast address is
selected from the administratively scoped IP multicast address
ranging from 239.242.x.y. The last two octets x and y are
generated based on the cluster group number. In the above
example cluster group number is 1985 and can be calculated as
follows: X= 1985/256 Y= 1985 - (256 *x)The system administrator can override the default multicast address by a unique address for their environment. |
TTL is the time to live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets. |
Enter "Yes" to enter the IP unicast addresses for the remote nodes of the cluster, which are not reachable using IP multicast address. |
In this example, 10.0.1.2 is the IP unicast address for the
node ORCHID. Although, the IP multicast is selected, ORCHID's IP
address is entered because the IP multicast connectivity between
SITE A and SITE B is presumed to be not existing in this
example. Note: Enter the list of IP address of the cluster. All
the information entered in[4],[6],[7] and [9] are entered into
the SYS$SYSTEM:PE$IP_CONFIG.DAT file. The PE$IP_CONFIG.DAT file
is generated as shown in the following example. Also, to allow
the remote node to join the cluster, the Unicast list in the
PE$IP_CONFIG.DAT on the local node must contain the IP address
of the remote node. In this example, TULIP must have ORCHID's IP
address and ORCHID must have TULIP's IP
address. ! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL-2008 14:14:06.57 multicast_address=239.242.7.193 ttl=32 udp_port=49152 unicast=10.0.1.2 |
CLUSTER_CONFIG_LAN.COM invokes TCPIP$CONFIG.COM to configure the IP interfaces used for cluster communication. Select the core environment option. Assuming that TCP/IP is already configured, the node can be pinged from outside the subnet. |
In the configuration option for the selected address, select
option 4 to add to IPCI database. The interface information
along with the default route is entered in the TCPIP$CLUSTER.DAT
as shown:
interface=IE0,EIA0,10.0.2.2,255.255.254.0 default_route=10.0.2.1 |
Step 2. Configuring Node ORCHID as a Cluster Member
ORCHID$ @SYS$MANAGER:CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". This IA64 node is not currently a cluster member. MAIN Menu 1. ADD ORCHID to existing cluster, or form a new cluster. 2. MAKE a directory structure for a new root on a system disk. 3. DELETE a root from a system disk. 4. EXIT from this procedure. Enter choice [4]: 1Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N What is the node's SCS node name? ORCHID IA64 node, using LAN/IP for cluster communications. PEDRIVER will be loaded. No other cluster interconnects are supported for IA64 nodes. Enter this cluster's group number: 1985 Enter this cluster's password: Re-enter this cluster's password for verification: ENABLE IP for cluster communications (Y/N)? Y
UDP port number to be used for Cluster Communication over IP[49152]?[Return]
Enable IP multicast for cluster communication(Y/N)[Y]? Y [Return]
What is IP the multicast address[239.242.7.193]? [Return]
What is the TTL (time to live) value for IP multicast packets [32]? [Return]
Do you want to enter unicast address(es)(Y/N)[Y]?[Return]
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]?[Return]
******************************************************************** Cluster Communications over IP has been enabled. Now CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure. Pleas select the IP interfaces to be used for Cluster Communications over IP (IPCI). This can be done selecting "Core Environment" option from the main menu followed by the "Interfaces" option. You may also use this opportunity to configure other aspects. ********************************************************************* Press Return to continue ... TCP/IP Network Configuration Procedure This procedure helps you define the parameters required to run TCP/IP Services for OpenVMS on this system. %TCPIP-I-IPCI, TCP/IP Configuration is limited to IPCI. -TCPIP-I-IPCI, Rerun TCPIP$CONFIG after joining the cluster. TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=Not Configured, Active=nodeg Configuration options: 0 - Set The Target Node (Current Node: ORCHID) 1 - IE0 Menu (EIA0: TwistedPair 100mbps) 2 - IE1 Menu (EIB0: TwistedPair 100mbps) [E] - Exit menu Enter configuration option: 1
* IPCI Address Configuration * Only IPCI addresses can be configured in the current environment. After configuring your IPCI address(es) it will be necessary to run TCPIP$CONFIG once your node has joined the cluster. IPv4 Address may be entered with CIDR bits suffix. E.g. For a 16-bit netmask enter 10.0.1.1/16 Enter IPv4 Address []:10.0.1.2/24
Default netmask calculated from class of IP address: 255.0.0.0 IPv4 Netmask may be entered in dotted decimal notation, (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16) Enter Netmask or CIDR bits [255.0.0.0]: 255.255.254.0
Requested configuration: Node : ORCHID Interface: IE0 IPCI : Yes Address : 10.0.1.2/23 Netmask : 255.255.254.0 (CIDR bits: 23) * Is this correct [YES]: Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT;
TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=Not Configured, Active=ORCHID Configuration options: 0 - Set The Target Node (Current Node: ORCHID) 1 - IE0 Menu (EIA0: TwistedPair 100mbps) 2 - 10.0.1.2 /23 *noname* IPCI 3 - IE1 Menu (EIB0: TwistedPair 100mbps) [E] - Exit menu Enter configuration option: E
Enter your Default Gateway address []: 10.0.1.1
* The default gateway will be: 10.0.1.1. Correct [NO]: YES Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT; TCPIP-I-IPCIDONE, Finished configuring IPCI address information
Will ORCHID be a boot server [Y]? [Return]
TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes. Use SYS$MANAGER:TCPIP$CONFIG.COM on ORCHID to enable BOOTP and TFTP services after ORCHID has booted into the cluster. Enter a value for ORCHID's ALLOCLASS parameter [7]: Does this cluster contain a quorum disk [N]? [Return] The EXPECTED_VOTES system parameter of members of a cluster indicates the total number of votes present when all cluster members are booted, and is used to determine the minimum number of votes (QUORUM) needed for cluster operation. EXPECTED_VOTES value for this cluster: 1 Warning: Setting EXPECTED_VOTES to 1 allows this node to boot without being able to see any other nodes in the cluster. If there is another instance of the cluster in existence that is unreachable via SCS but shares common drives (such as a Fibrechannel fabric)this may result in severe disk corruption. Do you wish to re-enter the value of EXPECTED_VOTES [Y]? N The use of a quorum disk is recommended for small clusters to maintain cluster quorum if cluster availability with only a single cluster node is a requirement. For complete instructions, check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. WARNING: ORCHID will be a voting cluster member. EXPECTED_VOTES for this and every other cluster member should be adjusted at a convenient time before a reboot. For complete instructions, check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. Execute AUTOGEN to compute the SYSGEN parameters for your configuration and reboot ORCHID with the new parameters. This is necessary before ORCHID can become a cluster member. Do you want to run AUTOGEN now [Y]? N Please run AUTOGEN to reboot ORCHID: ORCHID$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
Node ORCHID is currently a standalone, Integrity server node and is made as a member of a cluster. Only LAN or IP is used for cluster communication and no other interconnect is supported. |
Select IP for cluster communication in addition to LAN by entering "YES". The SYSGEN parameter, NISCS_USE_UDP is set to 1 and PEDRIVER uses IP in addition to LAN for cluster communication when the node is rebooted. |
The UDP port number to be used for cluster communication. The UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number and this port number must not be used by any other application. |
You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator to see if IP multicasting is enabled in your environment. |
Enter the IP multicast address for cluster, if IP multicasting
is enabled. By default, the IP multicast address is selected
from the administratively scoped IP multicast address range of
239.242.x.y. The last two octets x and y are generated based on
the cluster group number. In the above example cluster group
number is 1985 and is calculates as
follows: X= 1985/256 Y= 1985 - (256 *x)The system administrator can override the default multicast address by a unique address for their environment. |
TTL is the time to live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets. |
Enter "Yes" to enter the IP unicast address for remote nodes of the cluster which are not reachable using IP multicast address. |
In this example, 10.0.2.2 is the IP unicast address of node TULIP. Although, IP multicast is selected, TULIP's IP address is entered because the IP multicast connectivity between SITE A and SITE B is presumed to be non-existent in this example. NOTE: Enter the list of IP unicast address of the cluster. All the information entered in [2], [3], [5], [6], and [7] are entered in the SYS$SYSTEM:PE$IP_CONFIG.DAT file. The PE$IP_CONFIG.DAT file is generated as shown in following example. Also, the Unicast list in PE$IP_CONFIG.DAT in the local node should contain the remote node IP address for the local node to allow the remote node to join the cluster. In this example, ORCHID must have TULIP's IP address and TULIP
must have ORCHID's IP address. SYSTEM:PE$IP_CONFIG.DAT in node
ORCHID: ! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL- 2008 14:14:06.57 multicast_address=239.242.7.193 ttl=32 udp_port=49152 unicast=10.0.2.2 |
CLUSTER_CONFIG_LAN.COM invokes TCPIP$CONFIG.COM to configure the IP interfaces used for cluster communication. Currently, ORCHID is a standalone node, when TCPIP$CONFIG is invoked by the CLUSTER_CONFIG_LAN procedure, TCP/IP configuration is limited to IPCI. The interface, IE0 is selected for enabling cluster communications. Note: TCPIP$CONFIG must be invoked after joining the cluster for other TCP/IP configuration, such as, FTP, TELNET. |
The default gateway address for the interface IE0 is entered. Only one default gateway address is allowed for Cluster over IP communication. |
After the interface and default gateway are selected,
TCPIP$CONFIG updates the TCPIP$CLUSTER.DAT with the default
route or gateway information. This also completes the
TCPIP$CONFIG required for cluster communications using IP. The
interface information along with the default route is entered in
the TCPIP$CLUSTER.DAT file as shown:
interface=IE0,EIA0,10.0.1.2,255.255.254.0 default_route=10.0.1.1 |
8.2.3.2. Adding a new Node to a Cluster over IP
This section describes how to add a new node, JASMIN to an existing two-node cluster. Nodes, ORCHID and TULIP are currently members of a two-node cluster, which are at SITE A and SITE B. For more information about configuring a node with IP as interconnect, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”. Node JASMIN is currently a standalone node at SITE C with IP connectivity to both SITE A and SITE B.
Step 1. Ensuring IP Connectivity
Ensure that the IP connectivity between the node JASMIN and the nodes ORCHID and TULIP is working fine. Use the TCP/IP PING utility to test the IP connectivity between JASMIN and other nodes, ORCHID and TULIP.
If PING fails, set up TCP/IP configuration properly so that node JASMIN can ping both ORCHID and TULIP.
Step 2. Executing the CLUSTER_CONFIG_LAN.COM
Do you want to enter unicast address(es)(Y/N)[Y]?[Return]
What is the unicast address[Press [RETURN] to end the list]? 10.0.3.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2
What is the unicast address[Press [RETURN] to end the list]? [Return]
SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN shown below
! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL-2008 14:14:06.57
multicast_address=239.242.7.193
ttl=32
udp_port=49152
unicast=10.0.3.2
Unicast=10.0.2.2
Unicast=10.0.1.2
Enter the IP address of JASMIN, ORCHID and TULIP while configuring
the node JASMIN. NoteThe unicast list must be consistent in all nodes of the cluster. Hence, while entering the unicast list in JASMIN, enter the IP addresses of all the three nodes of the cluster (that is, JASMIN, ORCHID and TULIP). You can also enter the local nodes IP addresses along with the Unicast list as it facilitates system management. |
Step 3. Completing the Configuration Procedure Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP” for more details.
Step 4. Updating the PE$IP_CONFIG.DAT File To ensure that the nodes join the cluster, it is required to have PE$IP_CONFIG.DAT consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes, ORCHID, and TULIP.
Step 5. Refreshing the Unicast List
$MC SCACP RELOAD
Note
The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members of the SYS$SYSTEM:PE$IP_CONFIG.DAT file.
Step 6. Running AUTOGEN and Rebooting the Node
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
8.2.3.3. Adding a new Node to a Cluster over IP with a Shared System Disk
This section describes how to add a new node JASMIN that has a shared system disk of TULIP. ORCHID and TULIP are currently members of two-node cluster which are at SITE A and SITE B.
Step 1. Obtaining the Interface Information
Node JASMIN is an OpenVMS Alpha node and is directly connected to the system disk of one of the node TULIP. In this configuration, Node JASMIN is connected in network, but not yet booted.
P00>>>SHOW DEVICE dga5245.1003.0.3.0 $1$DGA5245 COMPAQ HSV110 (C)COMPAQ 3028 dga5245.1004.0.3.0 $1$DGA5245 COMPAQ HSV110 (C)COMPAQ 3028 dga5890.1001.0.3.0 $1$DGA5890 COMPAQ HSV110 (C)COMPAQ 3028 dga5890.1002.0.3.0 $1$DGA5890 COMPAQ HSV110 (C)COMPAQ 3028 dka0.0.0.2004.0 DKA0 COMPAQ BD03685A24 HPB7 dka100.1.0.2004.0 DKA100 COMPAQ BD01864552 3B08 dka200.2.0.2004.0 DKA200 COMPAQ BD00911934 3B00 dqa0.0.0.15.0 DQA0 HL-DT-ST CD-ROM GCR-8480 2.11 dva0.0.0.1000.0 DVA0 eia0.0.0.2005.0 EIA0 00-06-2B-03-2D-7D pga0.0.0.3.0 PGA0 WWN 1000-0000-c92a-78e9 pka0.7.0.2004.0 PKA0 SCSI Bus ID 7 pkb0.6.0.2.0 PKB0 SCSI Bus ID 6 5.57 P00>>>
From the output, the interface will be EIA0 on which the IP address will be configured and can be used for cluster formation.
shell>fs0: fs0:\> cd efi fs0:\EFI> cd vms fs0:\EFI\VMS> vms_show device VMS: EIA0 00-30-6E-F3-EC-6E EFI: Acpi(HWP0002,0)/Pci(3|0)/Mac(00306EF3EC6E) VMS: DKA100 HP 36.4GST336754LC HPC2 EFI: Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun1,Lun0) VMS: DKA0 HP 36.4GMAS3367NC HPC3 X8_3_XBJL EFI: fs0: Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0) VMS: EWA0 00-30-6E-F3-3C-28 EFI: Acpi(HWP0002,100)/Pci(2|0)/Mac(00306EF33C28) fs0:\EFI\VMS>
From the output, the interface will be EIA0. Here fs0: is the partition of the shared system disk.
Step 2. Executing CLUSTER_CONFIG_LAN.COM
TULIP$ @SYS$SYSROOT:[SYSMGR]CLUSTER_CONFIG_LAN.COM;1 Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an Alpha System DECnet Phase IV is installed on this node. Alpha satellites will use LANCP, not DECnet, for MOP downline loading. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". TULIP is an Alpha system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an Alpha node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for Tulip. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 1 This ADD function will add a new Alpha node to the cluster. WARNING: If the node being added is a voting member, EXPECTED_VOTES for every cluster member must be adjusted. For complete instructions check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. CAUTION: If this cluster is running with multiple system disks and common system files will be used, please, do not proceed unless appropriate logical names are defined for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. Do you want to continue [Y]?Y Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y Will the node be a satellite [Y]? N What is the node's SCS node name? JASMIN What is the node's SCSSYSTEMID number? 14487 Will JASMIN be a boot server [Y]?Y This procedure will now ask you for the device name of JASMIN's system root. The default device name (DISK$TULIPSYS:) is the logical volume name of SYS$SYSDEVICE:. What is the device name for JASMIN's system root [default DISK$TULIPSYS:]? What is the name of JASMIN's system root [SYS3]?SYS3 Creating directory tree SYS3 ... System root SYS3 created ENABLE IP for cluster communications (Y/N)? Y UDP port number to be used for Cluster Communication over IP[49152]?[Return] Enable IP multicast for cluster communication(Y/N)[Y]?Y What is the IP multicast address[224.0.0.3]? [Return] What is the TTL (time to live) value for IP multicast packets [1] ? [Return] Do you want to enter unicast address(es)(Y/N)[Y]?Y What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2 What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2 What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3 What is the unicast address[Press [RETURN] to end the list]? [Return] ***************************************************************** Cluster Communications over IP has been enabled. Now CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure. Please select the IP interfaces to be used for Cluster Communications over IP (IPCI). This can be done selecting "Core Environment" option from the main menu followed by the "Interfaces" option. You may also use this opportunity to configure other aspects. **************************************************************** Press Return to continue ... Checking TCP/IP Services for OpenVMS configuration database files. TCP/IP Services for OpenVMS Configuration Menu Configuration options: 1 - Core environment 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: 1 TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: 2 TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=TULIP, Active=TULIP Configuration options: 0 - Set The Target Node (Current Node: TULIP) 1 - WE0 Menu (EWA0: TwistedPair 1000mbps) 2 - 10.0.2.2/8 Tulip Configured,IPCI 3 - WE1 Menu (EWB0: TwistedPair 1000mbps) 4 - WE2 Menu (EWC0: TwistedPair 1000mbps) 5 - WE3 Menu (EWD0: TwistedPair 1000mbps) 6 - WE4 Menu (EWE0: TwistedPair 1000mbps) 7 - WE5 Menu (EWF0: TwistedPair 1000mbps) 8 - WE6 Menu (EWG0: Multimode 10000mbps) 9 - WE7 Menu (EWH0: TwistedPair 1000mbps) 10 - IE0 Menu (EIA0: TwistedPair 100mbps) 11 - IE1 Menu (EIB0: TwistedPair 100mbps) Enter configuration option or press ENTER key to continue: 0Enter name of node to manage [TULIP]: JASMIN JASMIN is not currently a cluster member. * Continue configuring JASMIN [NO]: Y
Enter system device for JASMIN [$10$DGA165:]:
Enter system root for JASMIN []: SYS3
TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=Not Configured Configuration options: 0 - Set The Target Node (Current Node: JASMIN - $10$DGA165:[sys3.]) A - Add an Interface [E] - Exit menu Enter configuration option: A Enter controller name (e.g. EIA or EWC, etc): [ENTER when done] EIA
Controller Name : EIA TCP/IP Interface Name : IE0 * Is this correct [NO]: Y Interface Menu: IE0 TCP/IP Services for OpenVMS Interface IE0 Configuration Menu (Node: JASMIN) Configuration options: 1 - Add a primary address on IE0 2 - Add an alias address on IE0 3 - Enable DHCP client to manage address on IE0 [E] - Exit menu Enter configuration option: 1
* Is this address used by Clusters over IP (IPCI) [NO]: Y
IPv4 Address may be entered with CIDR bits suffix. E.g. For a 16-bit netmask enter 10.0.1.1/16 Enter IPv4 Address []: 10.0.2.3 Default netmask calculated from class of IP address: 255.0.0.0 IPv4 Netmask may be entered in dotted decimal notation, (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16) Enter Netmask or CIDR bits [255.0.0.0]: Enter hostname []: JASMIN Requested configuration: Node : JASMIN Interface: IE0 IPCI : Yes Address : 10.0.2.3/8 Netmask : 255.0.0.0 (CIDR bits: 8) Hostname : JASMIN * Is this correct [YES]:Y Added hostname JASMIN (10.0.2.3) to host database NOTE: The system hostname is not configured. It will now be set to jasmin (10.0.2.3). This can be changed later via the Interface Configuration Menu. Updated system hostname in configuration database Added address IE0:10.0.2.3 to configuration database Updated Interface in IPCI configuration file: $10$DGA165:[SYS3.SYSEXE]TCPIP$CLUSTER.DAT; Updated Default Route in IPCI configuration file: $10$DGA165:[SYS3.SYSEXE]TCPIP$CLUSTER.DAT; TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=JASMIN Configuration options: 0 - Set The Target Node (Current Node: JASMIN - $10$DGA165:[sys3.] 1 - IE0 Menu (EIA0:) 2 - 10.0.2.3/8 JASMIN Configured,IPCI I - Information about your configuration A - Add an Interface [E] - Exit menu Enter configuration option: TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: TCP/IP Services for OpenVMS Configuration Menu Configuration options: 1 - Core environment 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN's root shown below ! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05 multicast_address=224.0.0.3 ttl=1 udp_port=49152 unicast=10.0.2.3 unicast=10.0.2.2 unicast=10.0.1.2 SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node JASMIN's root shown below interface=IE0,EIA0,10.0.2.3,255.0.0.0 default_route=16.116.40.1
In the TCP/IP configuration, select option 0 to set the target node to JASMIN, which is the new node added to the cluster. |
Enter the controller information on which IP will be configured for cluster traffic. This is the controller information that has been obtained from the console of the machine JASMIN as explained in the beginning of the configuration. |
Step 3. Completing the Configuration Procedure
Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure. For more information, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”.
Step 4. Updating the PE$IP_CONFIG.DAT file
To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes, ORCHID and TULIP.
Step 5. Refreshing the Unicast list
$ MC SCACP RELOAD
Note
The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members SYS$SYSTEM:PE$IP_CONFIG.DAT file.
Step 6. Running AUTOGEN and Rebooting the Node
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
8.2.3.4. Adding an Integrity server Satellite node to a Cluster over IP
Note
For both Alpha and Integrity server satellite nodes, the satellite node and its boot server must exist in the same LAN segment.
Step 1. Selecting the Interface for Satellite Booting
P00>>>SHOW DEVICE dga5245.1003.0.3.0 $1$DGA5245 COMPAQ HSV110 (C)COMPAQ 3028 dga5245.1004.0.3.0 $1$DGA5245 COMPAQ HSV110 (C)COMPAQ 3028 dga5890.1001.0.3.0 $1$DGA5890 COMPAQ HSV110 (C)COMPAQ 3028 dga5890.1002.0.3.0 $1$DGA5890 COMPAQ HSV110 (C)COMPAQ 3028 dka0.0.0.2004.0 DKA0 COMPAQ BD03685A24 HPB7 dka100.1.0.2004.0 DKA100 COMPAQ BD01864552 3B08 dka200.2.0.2004.0 DKA200 COMPAQ BD00911934 3B00 dqa0.0.0.15.0 DQA0 HL-DT-ST CD-ROM GCR-8480 2.11 dva0.0.0.1000.0 DVA0 eia0.0.0.2005.0 EIA0 00-06-2B-03-2D-7D pga0.0.0.3.0 PGA0 WWN 1000-0000-c92a-78e9 pka0.7.0.2004.0 PKA0 SCSI Bus ID 7 pkb0.6.0.2.0 PKB0 SCSI Bus ID 6 5.57 P00>>>
Note
The Alpha console uses the MOP protocol for network load of satellite systems. Because the MOP protocol is non-routable, the satellite boot server or servers and all satellites booting from them must reside in the same LAN. In addition, the boot server must have at least one LAN device enabled for cluster communications to permit the Alpha satellite nodes to access the system disk.
Shell> lanaddress LAN Address Information LAN Address Path ----------------- ---------------------------------------- Mac(00306E4A133F) Acpi(HWP0002,0)/Pci(3|0)/Mac(00306E4A133F)) *Mac(00306E4A02F9) Acpi(HWP0002,100)/Pci(2|0)/Mac(00306E4A02F9)) Shell>
Assuming that the interface which is active is EIA0, configure the satellite with EIA0, if it does not boot with EIA0 then try with EWA0 subsequently.
Step 2. Executing CLUSTER_CONFIG_LAN.COM
TULIP$ @SYS$SYSROOT:[SYSMGR]CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". TULIP is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for TULIP. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: This ADD function will add a new IA64 node to the cluster. WARNING: If the node being added is a voting member, EXPECTED_VOTES for every cluster member must be adjusted. For complete instructions check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. CAUTION: If this cluster is running with multiple system disks and common system files will be used, please, do not proceed unless appropriate logical names are defined for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. Do you want to continue [Y]? Y Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N Will the node be a satellite [Y]? [Return] What is the node's SCS node name? JASMIN What is the node's SCSSYSTEMID number? 25482 WARNING: DECnet is not running. No DECnet databases will be updated with information on JASMIN. Does JASMIN need to be registered in the DECnet namespace [N]?[Return] What is the Cluster Alias fullname? This procedure will now ask you for the device name of JASMIN's system root. The default device name (DISK$TULIPSYS:) is the logical volume name of SYS$SYSDEVICE:. What is the device name for JASMIN's system root [default DISK$TULIPSYS:]? [Return] What is the name of JASMIN's system root [SYS14]? [Return] What is JASMIN's LAN adapter hardware address? 00-30-6E-4A-02-F9What is JASMIN's TCP/IP address [10.0.2.3]? [Return]
What is JASMIN's TCP/IP gateway or gateways (leave blank if none)? 10.0.2.1
What is JASMIN's TCP/IP network mask [255.255.255.0]? 255.255.254.0
NOTE: Make sure to set the VMS_FLAGS console variable to 0,200000 on node JASMIN so it will use the memory-disk method to boot as a satellite. The command to update this variable from the console EFI shell of JASMIN is: set vms_flags "0,200000" Allow conversational bootstraps on JASMIN [N]? [Return] The following workstation windowing options are available: 1. No workstation software 2. DECwindows Workstation Software Enter choice [1]: Creating directory tree SYS14 ... System root SYS14 created ENABLE IP for cluster communications (Y/N)? Y
UDP port number to be used for Cluster Communication over IP[49152]? [Return]
Enable IP multicast for cluster communication(Y/N)[Y]? Y
What is IP the multicast address[224.0.0.3]? [Return] What is the TTL (time to live) value for IP multicast packets [1] ? 32]? [Return]
Do you want to enter unicast address(es)(Y/N)[Y]? Y
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3 What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2 What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2 What is the unicast address[Press [RETURN] to end the list]? [Return] ***************************************************************** Cluster Communications over IP has been enabled. Now CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure. Please select the IP interfaces to be used for Cluster Communications over IP (IPCI). This can be done selecting "Core Environment" option from the main menu followed by the "Interfaces" option. You may also use this opportunity to configure other aspects. **************************************************************** Press Return to continue ... Checking TCP/IP Services for OpenVMS configuration database files. Configuration options: 1 - Core environment 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: 1 TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: 2 TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=[], Active=[] Configuration options: 0 - Set The Target Node (Current Node: TULIP) 1 - IE0 Menu (EIA0: TwistedPair 100mbps) 2 - 15.146.235.222/23 *noname* Configured 3 - 15.146.235.254/23 [] IPCI 4 - IE1 Menu (EIB0: TwistedPair 100mbps) 5 - 15.146.235.222/23 *noname* Configured,Active I - Information about your configuration [E] - Exit menu Enter configuration option: 0
Enter name of node to manage [TULIP]: JASMIN JASMIN is not currently a cluster member. * Continue configuring JASMIN [NO]: YES
Enter system device for JASMIN [DSA2:]:
Enter system root for JASMIN []: SYS14
TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=JASMIN Configuration options: 0 - Set The Target Node (Current Node: JASMIN - DSA2:[SYS14] A - Add an Interface [E] - Exit menu Enter configuration option: a Enter controller name (e.g. EIA or EWC, etc): [ENTER when done] EIA
Controller Name : EIA TCP/IP Interface Name : IE0 * Is this correct [NO]: y Interface Menu:IE0 TCP/IP Services for OpenVMS Interface IE0 Configuration Menu (Node: JASMIN) Configuration options: 1 - Add a primary address on IE0 2 - Add an alias address on IE0 3 - Enable DHCP client to manage address on IE0 [E] - Exit menu Enter configuration option: 1
* Is this address used by Clusters over IP (IPCI) [NO]: Y
IPv4 Address may be entered with CIDR bits suffix. E.g. For a 16-bit netmask enter 10.0.1.1/16 Enter IPv4 Address []: 10.0.2.3 Default netmask calculated from class of IP address: 255.0.0.0 IPv4 Netmask may be entered in dotted decimal notation, (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16) Enter Netmask or CIDR bits [255.0.0.0]: Enter hostname []: JASMIN Requested configuration: Node : JASMIN Interface: IE0 IPCI : Yes Address : 10.0.2.3/8 Netmask : 255.0.0.0 (CIDR bits: 8) Hostname : jasmin * Is this correct [YES]: Added hostname jasmin (10.0.2.3) to host database NOTE: The system hostname is not configured. It will now be set to jasmin (10.0.2.3). This can be changed later via the Interface Configuration Menu. Updated system hostname in configuration database Added address IE1:10.0.2.3 to configuration database Updated Interface in IPCI configuration file: DSA2:[SYS14.SYSEXE]TCPIP$CLUSTER.DAT; Updated Default Route in IPCI configuration file: DSA2:[SYS14.SYSEXE]TCPIP$CLUSTER.DAT; TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=JASMIN Configuration options: 0 - Set The Target Node (Current Node: JASMIN - DSA2:[SYS14.]) 1 - IE1 Menu (EIB0:) 2 - 10.0.2.3/8 JASMIN Configured,IPCI I - Information about your configuration A - Add an Interface [E] - Exit menu Enter configuration option: TCP/IP Services for OpenVMS Core Environment Configuration Menu Configuration options: 1 - Domain 2 - Interfaces 3 - Routing 4 - BIND Resolver 5 - Time Zone A - Configure options 1 - 5 [E] - Exit menu Enter configuration option: E TCP/IP Services for OpenVMS Configuration Menu Configuration options: 1 - Core environment 2 - Client components 3 - Server components 4 - Optional components 5 - Shutdown TCP/IP Services for OpenVMS 6 - Startup TCP/IP Services for OpenVMS 7 - Run tests A - Configure options 1 - 4 [E] - Exit configuration procedure Enter configuration option: E SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN's root shown below ! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05 multicast_address=224.0.0.3 ttl=32 udp_port=49152 unicast=10.0.2.3 unicast=10.0.2.2 unicast=10.0.1.2 SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node JASMIN's root shown below interface=IE0,EIA0,10.0.2.3,255.0.0.0 default_route=16.116.40.1
Note
Assuming that the interface which is active is EIA, configure the satellite with EIA, if it does not boot with EIA then try with EIB subsequently. If the wrong interface name is given, satellite node fails with the messages during booting.
The UDP port number to be used for cluster communication. The UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number and this port number must not be used by any other application. |
Enter the IP multicast address for cluster, if IP multicasting is
enabled. By default, the IP multicast address is selected from the
administratively scoped IP multicast address range of 239.242.x.y.
The last two octets x and y are generated based on the cluster group
number. In the above example, the cluster group number is 1985 and
can be calculated as
follows: X= 1985/256 Y= 1985 - (256 *x) The system administrator can override the default multicast address with a unique address for their environment. The multicast address is modified based on the cluster group number or it can be added to .DAT file. |
TTL is the time-to-live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets. |
Enter "yes" to enter the IP Unicast address of remote nodes of the cluster, which are not reachable using IP multicast address. |
In the TCP/IP configuration, select option 0 to set the target node to JASMIN, which is the satellite node, and will be added to the cluster. |
Enter the controller information on which IP will be configured for cluster communication. The controller information is obtained from the console of JASMIN as explained in the beginning of the configuration. |
Step 3. Executing the CLUSTER_CONFIG_LAN.COM Procedure
Adjusting protection on DSA2:[SYS14.][SYSEXE]PE$IP_CONFIG.DAT;1 Will JASMIN be a disk server [N]? Y Enter a value for JASMIN's ALLOCLASS parameter [0]: 15 Updating BOOTP database with satellite information for JASMIN.. Size of pagefile for JASMIN [RETURN for AUTOGEN sizing]? A temporary pagefile will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary pagefile [10000]? [Return] Size of swap file for JASMIN [RETURN for AUTOGEN sizing]? [Return] A temporary swap file will be created until resizing by AUTOGEN. The default size below is arbitrary and may or may not be appropriate. Size of temporary swap file [8000]? [Return] NOTE: IA64 satellite node JASMIN requires DOSD if capturing the system state in a dumpfile is desired after a system crash. Will a local disk on JASMIN be used for paging and swapping (Y/N)? N If you specify a device other than DISK$TULIPSYS: for JASMIN's page and swap files, this procedure will create PAGEFILE_JASMIN.SYS and SWAPFILE_JASMIN.SYS in the <SYSEXE> directory on the device you specify. What is the device name for the page and swap files [DISK$TULIPSYS:]? [Return] %SYSGEN-I-CREATED, DSA2:<SYS14.SYSEXE>PAGEFILE.SYS;1 created %SYSGEN-I-CREATED, DSA2:<SYS14.SYSEXE>SWAPFILE.SYS;1 created The configuration procedure has completed successfully.
The node JASMIN is configured to join the cluster. After the first boot of JASMIN, AUTOGEN.COM will run automatically.
Step 4. Updating the PE$IP_CONFIG.DAT File
To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN's root to the other nodes, ORCHID and TULIP.
Step 5. Refreshing the Unicast list
$MC SCACP RELOAD
Note
The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members of the SYS$SYSTEM:PE$IP_CONFIG.DAT file.
Step 6. Running AUTOGEN and Rebooting the Node
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
8.2.3.5. Adding an Integrity server Node to a Cluster over IP with Logical LAN Failover set
This section describes how to add a node, ORCHID to an existing two-node cluster, JASMIN and TULIP. The Logical LAN failover set is created and configured on ORCHID. ORCHID can survive failure if a local LAN card fails and it will switchover to other interface configured in the logical LAN failover set.
Step 1. Configuring the Logical LAN Failover set
$ MC LANCP LANCP>DEFINE DEVICE LLB/ENABLE/FAILOVER=(EIA0, EIB0))
%LLB0, Logical LAN event at 2-SEP-2008 14:52:50.06 %LLB0, Logical LAN failset device created
Step 2: Executing CLUSTER_CONFIG_LAN
ORCHID$@SYS$MANAGER:CLUSTER_CONFIG_LAN Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". This IA64 node is not currently a cluster member. MAIN Menu 1. ADD ORCHID to existing cluster, or form a new cluster. 2. MAKE a directory structure for a new root on a system disk. 3. DELETE a root from a system disk. 4. EXIT from this procedure. Enter choice [4]: 1 Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? n What is the node's SCS node name? orchid IA64 node, using LAN/IP for cluster communications. PEDRIVER will be loaded. No other cluster interconnects are supported for IA64 nodes. Enter this cluster's group number: 1985 Enter this cluster's password: Re-enter this cluster's password for verification: ENABLE IP for cluster communications (Y/N)? Y UDP port number to be used for Cluster Communication over IP[49152]? [Return] Enable IP multicast for cluster communication(Y/N)[Y]? [Return] What is IP the multicast address[239.242.7.193]? 239.242.7.193 What is the TTL (time to live) value for IP multicast packets [32]? [Return] Do you want to enter unicast address(es)(Y/N)[Y]? [Return] What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2 What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3 What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2 What is the unicast address[Press [RETURN] to end the list]? [Return] ***************************************************************** Cluster Communications over IP has been enabled. Now CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure. Please select the IP interfaces to be used for Cluster Communications over IP (IPCI). This can be done selecting "Core Environment" option from the main menu followed by the "Interfaces" option. You may also use this opportunity to configure other aspects. **************************************************************** Press Return to continue ... TCP/IP Network Configuration Procedure This procedure helps you define the parameters required to run TCP/IP Services for OpenVMS on this system. %TCPIP-I-IPCI, TCP/IP Configuration is limited to IPCI. -TCPIP-I-IPCI, Rerun TCPIP$CONFIG after joining the cluster. TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=Not Configured, Active=nodeg Configuration options: 0 - Set The Target Node (Current Node: ORCHID) 1 - LE0 Menu (LLA0: TwistedPair 100mbps) 2 - IE1 Menu (EIB0: TwistedPair 100mbps) [E] - Exit menu Enter configuration option: 1 * IPCI Address Configuration * Only IPCI addresses can be configured in the current environment. After configuring your IPCI address(es) it will be necessary to run TCPIP$CONFIG once your node has joined the cluster. IPv4 Address may be entered with CIDR bits suffix. E.g. For a 16-bit netmask enter 10.0.1.1/16 Enter IPv4 Address []:10.0.1.2 Default netmask calculated from class of IP address: 255.0.0.0 IPv4 Netmask may be entered in dotted decimal notation, (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16) Enter Netmask or CIDR bits [255.0.0.0]: 255.255.255.0 Requested configuration: Node : ORCHID Interface: IE0 IPCI : Yes Address : 10.0.1.2/24 Netmask : 255.255.254.0 (CIDR bits: 23) * Is this correct [YES]: Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER. DAT; TCP/IP Services for OpenVMS Interface & Address Configuration Menu Hostname Details: Configured=Not Configured, Active=nodeg Configuration options: 0 - Set The Target Node (Current Node: ORCHID) 1 - LE0 Menu (LLA0: TwistedPair 100mbps) 2 - 10.0.1.2/24 ORCHID IPCI 3 - IE1 Menu (EIB0: TwistedPair 100mbps) [E] - Exit menu Enter configuration option: E Enter your Default Gateway address []: 10.0.1.1 * The default gateway will be: 10.0.1.1 Correct [NO]: YES Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT; TCPIP-I-IPCIDONE, Finished configuring IPCI address information. SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node ORCHID's root shown below ! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05 multicast_address=239.242.7.193 ttl=32 udp_port=49152 unicast=10.0.2.3 unicast=10.0.2.2 unicast=10.0.1.2 SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node ORCHID's root shown below interface=LE1,LLB0,10.0.1.2,255.0.0.0 default_route=10.0.1.1
Step 3. Completing the Configuration Procedure
Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure. For more information, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”.
Step 4. Updating the PE$IP_CONFIG.DAT file
To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes ORCHID and TULIP.
Step 5. Refreshing the Unicast list
$MC SCACP RELOAD
You can also use SYSMAN and run the command cluster wide.
Step 6. Running AUTOGEN and Rebooting the Node
ORCHID$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT
8.2.4. Adding a Quorum Disk
IF... |
THEN... |
---|---|
Other cluster nodes are already enabled as quorum disk watchers. |
Perform the following steps:
|
The cluster does not contain any quorum disk watchers. |
Perform the following steps:
|
8.3. Removing Computers
- Determine whether removing a member will cause you to lose quorum. Use the SHOW CLUSTER command to display the CL_QUORUM and CL_VOTES values.
IF removing members...
THEN...
Will cause you to lose quorum
Perform the steps in the following list:
Caution: Do not perform these steps until you are ready to reboot the entire OpenVMS Cluster system. Because you are reducing quorum for the cluster, the votes cast by the node being removed could cause a cluster partition to be formed.Reset the EXPECTED_VOTES parameter in the AUTOGEN parameter files and current system parameter files (see Section 8.6.1, “Updating Parameter Files”).
Shut down the cluster (see Section 8.6.2, “Shutting Down the Cluster”), and reboot without the node that is being removed.
Note: Be sure that you do not specify an automatic reboot on that node.
Will not cause you to lose quorum
Proceed as follows:Perform an orderly shutdown on the node being removed by invoking the SYS$SYSTEM:SHUTDOWN.COM command procedure (described in Section 8.6.3, “Shutting Down a Single Node”).
If the node was a voting member, use the DCL command SET CLUSTER/EXPECTED_VOTES to reduce the value of quorum.
Reference: Refer also to Section 10.11, “Restoring Cluster Quorum” for information about adjusting expected votes.
Invoke CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM on an active OpenVMS Cluster computer and select the REMOVE option.
- Use the information in Table 8.6, “Preparing to Remove Computers from an OpenVMS Cluster ” to determine whether additional actions are required.
Table 8.6. Preparing to Remove Computers from an OpenVMS Cluster IF...
THEN...
You are removing a voting member.
You must, after the REMOVE function completes, reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”.
The page and swap files for the computer being removed do not reside on the same disk as the computer's root directory tree.
The REMOVE function does not delete these files. It displays a message warning that the files will not be deleted, as in Example 8.6, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Remove a Satellite with Local Page and Swap Files”. If you want to delete the files, you must do so after the REMOVE function completes.
You are removing a computer from a cluster that uses DECdtm services.
Make sure that you have followed the step-by-step instructions in the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual. These instructions describe how to remove a computer safely from the cluster, thereby preserving the integrity of your data.
Note: When the REMOVE function deletes the computer's entire root directory tree, it generates OpenVMS RMS informational messages while deleting the directory files. You can ignore these messages.
8.3.1. Example
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 2 The REMOVE command disables a node as a cluster member. o It deletes the node's root directory tree. o If the node has entries in SYS$DEVICES.DAT, any port allocation class for shared SCSI bus access on the node must be re-assigned. If the node being removed is a voting member, EXPECTED_VOTES in each remaining cluster member's MODPARAMS.DAT must be adjusted. The cluster must then be rebooted. For instructions, see the "OpenVMS Cluster Systems" manual. CAUTION: The REMOVE command does not remove the node name from any network databases. Also, if a satellite has been set up for booting with multiple hardware addresses, the satellite's aliases are not cleared from the LANACP boot database. What is the node's SCS node name? GOMTHI Verifying BOOTP satellite node database... Verifying that $1$DKA0:[SYS10] is GOMTHI's root... Are you sure you want to remove node GOMTHI (Y/N)? Y WARNING: GOMTHI's page and swap files will not be deleted. They do not reside on $1$DKA0:. Deleting directory tree $1$DKA0:<SYS10...> %DELETE-I-FILDEL, $1$DKA0:<SYS10.SYS$I18N.LOCALES>SYSTEM.DIR;1 deleted (16 blocks) . . . . %DELETE-I-FILDEL, $1$DKA0:<SYS10>VPM$SERVER.DIR;1 deleted (16 blocks) %DELETE-I-TOTAL, 21 files deleted (336 blocks) %DELETE-I-FILDEL, $1$DKA0:<0,0>SYS10.DIR;1 deleted (16 blocks) System root $1$DKA0:<SYS10> deleted. Updating BOOTP database... Removing rights identifier for GOMTHI... The configuration procedure has completed successfully.
8.3.2. Removing a Quorum Disk
IF... |
THEN... |
---|---|
Other cluster nodes will still be enabled as quorum disk watchers. |
Perform the following steps:
|
All quorum disk watchers will be disabled. |
Perform the following steps:
|
8.4. Changing Computer Characteristics
As your processing needs change, you may want to add satellites to an existing OpenVMS Cluster, or you may want to change an OpenVMS Cluster that is based on one interconnect (such as the CI or DSSI interconnect, or HSC subsystem) to include several interconnects.
Table 8.8, “CHANGE Options of the Cluster Configuration Procedure” describes the operations you can accomplish when you select the CHANGE option from the main menu of the cluster configuration command procedure.
8.4.1. Preparation
You usually need to perform a number of steps before using the cluster configuration command procedure to change the configuration of your existing cluster.
Task |
Procedure |
---|---|
Add satellite nodes |
Perform these operations on the computer that will be
enabled as a cluster boot server:
|
Change an existing CI or DSSI cluster to include satellite nodes |
To enable cluster communications over the LAN
(Ethernet or FDDI) on all computers, and to enable one or
more computers as boot servers, proceed as follows:
|
Change an existing LAN-based cluster to include CI and DSSI interconnects |
Before performing the operations described here, be sure that the computers and HSC subsystems or RF disks you intend to include in your new configuration are correctly installed and checked for proper operation. The method you use to include CI and DSSI interconnects with an existing LAN-based cluster configuration depends on whether your current boot server is capable of being configured as a CI or DSSI computer. Note: The following
procedures assume that the system disk containing satellite
roots will reside on an HSC disk (for CI configurations) or
an RF disk (for DSSI configurations).
|
Convert a standalone computer to an OpenVMS Cluster computer |
Execute either CLUSTER_CONFIG_LAN.COM or
CLUSTER_CONFIG.COM on a standalone computer to perform
either of the following operations:
See Example 8.13, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Convert a Standalone Computer to a Cluster Boot Server”, which illustrates the use of CLUSTER_CONFIG.COM on standalone computer PLUTO to convert PLUTO to a cluster boot server. If your cluster uses DECdtm services, you must create a transaction log for the computer when you have configured it into your cluster. For step-by-step instructions on how to do this, see the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual. |
Enable or disable disk-serving or tape-serving functions |
After invoking either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to enable or disable the disk or tape serving functions, run AUTOGEN with the REBOOT option to reboot the local computer (see Section 8.6.1, “Updating Parameter Files”). |
Note: When the cluster configuration command procedure sets or changes values in MODPARAMS.DAT, the new values are always appended at the end of the file so that they override earlier values. You may want to edit the file occasionally and delete lines that specify earlier values.
8.4.2. Examples
Enable a computer as a disk server (Example 8.7, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Disk Server”).
Change a computer's ALLOCLASS value (Example 8.8, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change the Local Computer's ALLOCLASS Value”).
Enable a computer as a boot server (Example 8.9, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Boot Server”).
Specify a new hardware address for a satellite node that boots from a common system disk (Example 8.10, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change a Satellite's Hardware Address”).
Enable a computer as a tape server (Example 8.11, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Tape Server”).
Change a computer's TAPE_ALLOCLASS value (Example 8.12, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change the Local Computer's TAPE_ALLOCLASS Value”).
Convert a standalone computer to a cluster boot server (Example 8.13, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Convert a Standalone Computer to a Cluster Boot Server”).
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 5 Enter a value for BHAGAT's ALLOCLASS parameter [1]: The configuration procedure has completed successfully. BHAGAT has been enabled as a disk server. In MODPARAMS.DAT: MSCP_LOAD has been set to 1 MSCP_SERVE_ALL has been set to 2 Please run AUTOGEN to reboot BHAGAT: $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the OpenVMS Cluster Systems manual.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 7 Enter a value for BHAGAT's ALLOCLASS parameter [1]: 2 The configuration procedure has completed successfully. Since you have changed BHAGAT's ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the "OpenVMS Cluster Systems" manual. This includes running AUTOGEN for BHAGAT as shown below, before rebooting the cluster: $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the OpenVMS Cluster Systems manual.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 1 Enter a value for BHAGAT's ALLOCLASS parameter [1]: [Return] The configuration procedure has completed successfully. BHAGAT has been enabled as a boot server. Disk serving and LAN capabilities are enabled automatically. If BHAGAT was not previously set up as a disk server, please run AUTOGEN to reboot BHAGAT: $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the OpenVMS Cluster Systems manual.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 11 What is the node's SCS node name? gomthi Note: The current hardware address entry for GOMTHI is 00-30-6E-4C-BB-1A. What is GOMTHI's new LAN adapter hardware address? 00-30-6E-4C-BA-2A The configuration procedure has completed successfully.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 8 Enter a value for BHAGAT's TAPE_ALLOCLASS parameter [0]: [Return] Should BHAGAT serve any tapes it sees, local and remote [Y]? [Return] BHAGAT has been enabled as a tape server. In MODPARAMS.DAT, TMSCP_LOAD has been set to 1 TMSCP_SERVE_ALL has been set to 1 Please run AUTOGEN to reboot BHAGAT: $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the "OpenVMS Cluster Systems" manual.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 3 CHANGE Menu 1. Enable BHAGAT as a boot server. 2. Disable BHAGAT as a boot server. 3. Enable a quorum disk for BHAGAT. 4. Disable a quorum disk for BHAGAT. 5. Enable BHAGAT as a disk server. 6. Disable BHAGAT as a disk server. 7. Change BHAGAT's ALLOCLASS value. 8. Enable BHAGAT as a tape server. 9. Disable BHAGAT as a tape server. 10. Change BHAGAT's TAPE_ALLOCLASS value. 11. Change an IA64 satellite node's LAN adapter hardware address. 12. Enable Cluster Communication using IP on BHAGAT. 13. Disable Cluster Communication using IP on BHAGAT. 14. Change BHAGAT's shared SCSI port allocation class value. 15. Reset an IA64 satellite node's boot environment file protections. 16. Return to MAIN menu. Enter choice [16]: 10 Enter a value for BHAGAT's TAPE_ALLOCLASS parameter [0]: 1 If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the "OpenVMS Cluster Systems" Manual. This includes running AUTOGEN for BHAGAT as shown below, before rebooting the cluster: $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure the cluster, using the procedure described in the OpenVMS Cluster Systems manual.
$ @CLUSTER_CONFIG_LAN.COM IA64 platform support is in procedure CLUSTER_CONFIG_LAN.COM. The currently running procedure, CLUSTER_CONFIG.COM, will call it for you. Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". This IA64 node is not currently a cluster member. MAIN Menu 1. ADD MOON to existing cluster, or form a new cluster. 2. MAKE a directory structure for a new root on a system disk. 3. DELETE a root from a system disk. 4. EXIT from this procedure. Enter choice [4]: 1 Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N What is the node's SCS node name? moon DECnet is running on this node. Even though you are configuring a LAN- based cluster, the DECnet database will provide some information and may be updated. Do you want to define a DECnet synonym [Y]? N IA64 node, using LAN for cluster communications. PEDRIVER will be loaded. No other cluster interconnects are supported for IA64 nodes. Enter this cluster's group number: 123 Enter this cluster's password: Re-enter this cluster's password for verification: Will MOON be a boot server [Y]? [Return] TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes. Use SYS$MANAGER:TCPIP$CONFIG.COM on MOON to enable BOOTP and TFTP services after MOON has booted into the cluster. Enter a value for MOON's ALLOCLASS parameter [0]:[Return] Does this cluster contain a quorum disk [N]? [Return] The EXPECTED_VOTES system parameter of members of a cluster indicates the total number of votes present when all cluster members are booted, and is used to determine the minimum number of votes (QUORUM) needed for cluster operation. EXPECTED_VOTES value for this cluster: 1 Warning: Setting EXPECTED_VOTES to 1 allows this node to boot without being able to see any other nodes in the cluster. If there is another instance of the cluster in existence that is unreachable via SCS but shares common drives (such as a Fibrechannel fabric) this may result in severe disk corruption. Do you wish to re-enter the value of EXPECTED_VOTES [Y]? N The use of a quorum disk is recommended for small clusters to maintain cluster quorum if cluster availability with only a single cluster node is a requirement. For complete instructions, check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. WARNING: MOON will be a voting cluster member. EXPECTED_VOTES for this and every other cluster member should be adjusted at a convenient time before a reboot. For complete instructions, check the section on configuring a cluster in the "OpenVMS Cluster Systems" manual. Execute AUTOGEN to compute the SYSGEN parameters for your configuration and reboot MOON with the new parameters. This is necessary before MOON can become a cluster member. Do you want to run AUTOGEN now [Y]? [Return] Running AUTOGEN -- Please wait. %AUTOGEN-I-BEGIN, GETDATA phase is beginning. . . .
8.5. Creating a Duplicate System Disk
As you continue to add Integrity servers running on a common Integrity common system disk, or Alpha computers running on an Alpha common system disk, you eventually reach the disk's storage or I/O capacity. In that case, you want to add one or more common system disks to handle the increased load.
Reminder: Remember that a system disk cannot be shared between two architectures. Furthermore, you cannot create a system disk for one architecture from a system disk of a different architecture.
8.5.1. Preparation
Locate an appropriate scratch disk for use as an additional system disk.
Log in as system manager.
Invoke either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM and select the CREATE option.
8.5.2. Example
Prompts for the device names of the current and new system disks.
Backs up the current system disk to the new one.
Deletes all directory roots (except SYS0) from the new disk.
Mounts the new disk clusterwide.
$ @CLUSTER_CONFIG_LAN.COM Cluster/IPCI Configuration Procedure CLUSTER_CONFIG_LAN Version V2.84 Executing on an IA64 System DECnet-Plus is installed on this node. IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading. TCP/IP is installed and running on this node. Enter a "?" for help at any prompt. If you are familiar with the execution of this procedure, you may want to mute extra notes and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF". BHAGAT is an IA64 system and currently a member of a cluster so the following functions can be performed: MAIN Menu 1. ADD an IA64 node to the cluster. 2. REMOVE a node from the cluster. 3. CHANGE a cluster member's characteristics. 4. CREATE a duplicate system disk for BHAGAT. 5. MAKE a directory structure for a new root on a system disk. 6. DELETE a root from a system disk. 7. EXIT from this procedure. Enter choice [7]: 4 The CREATE function generates a duplicate system disk. o It backs up the current system disk to the new system disk. o It then removes from the new system disk all system roots. WARNING: Do not proceed unless you have defined appropriate logical names for cluster common files in SYLOGICALS.COM. For instructions, refer to the "OpenVMS Cluster Systems" manual. Do you want to continue [N]? Y This procedure will now ask you for the device name of the current system disk. The default device name (DISK$BHAGAT_SYS:) is the logical volume name of SYS$SYSDEVICE:. What is the device name of the current system disk [DISK$BHAGAT_SYS:]? What is the device name of the new system disk? . . .
8.6. Post-configuration Tasks
Some configuration functions, such as adding or removing a voting member or enabling or disabling a quorum disk, require one or more additional operations.
After running the cluster configuration procedure to... | You should... | |
---|---|---|
Add or remove a voting member |
Update the AUTOGEN parameter files and the current system parameter files for all nodes in the cluster, as described in Section 8.6.1, “Updating Parameter Files”. | |
Enable a quorum disk |
Perform the following steps:
Reference: See also Section 8.2.4, “Adding a Quorum Disk” for more information about adding a quorum disk. | |
Disable a quorum disk |
Perform the following steps: Caution: Do not perform these
steps until you are ready to reboot the entire OpenVMS Cluster
system. Because you are reducing quorum for the cluster, the
votes cast by the quorum disk being removed could cause cluster
partitioning.
| |
Quorum will not be lost |
THEN... | |
Quorum will not be lost |
Perform these steps:
| |
Quorum will be lost |
Shut down and reboot the entire cluster. | |
Reference: Cluster shutdown is described in Section 8.6.2, “Shutting Down the Cluster”. | ||
Reference: See also Section 8.3.2, “Removing a Quorum Disk” for more information about removing a quorum disk. | ||
Add a satellite node |
Perform these steps:
| |
Enable or disable the LAN or IP for cluster communications |
Update the current system parameter files and reboot the node on which you have enabled or disabled the LAN or IP (Section 8.6.1, “Updating Parameter Files”). | |
Change allocation class values |
Update the current system parameter files and shut down and reboot the entire cluster (Sections 8.6.1 and 8.6.2). | |
Change the cluster group number or password |
Shut down and reboot the entire cluster (Sections 8.6.2 and 8.6.7). |
8.6.1. Updating Parameter Files
The cluster configuration command procedures (CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM) can be used to modify parameters in the AUTOGEN parameter file for the node on which it is run.
In some cases, such as when you add or remove a voting cluster member, or when you enable or disable a quorum disk, you must update the AUTOGEN files for all the other cluster members.
Method |
Description |
---|---|
Update MODPARAMS.DAT files |
Edit MODPARAMS.DAT in all cluster members' [SYS
For example, if you add a voting member or if you enable a quorum disk, you must increment the value by the number of votes assigned to the new member (usually 1). If you add a voting member with one vote and enable a quorum disk with one vote on that computer, you must increment the value by 2. |
Update AGEN$ files |
Update the parameter settings in the appropriate AGEN$
include files:
Reference: These files are described in Section 8.2.2, “Common AUTOGEN Parameter Files”. |
You must also update the current system parameter files (IA64VMSSYS.PAR or ALPHAVMSSYS.PAR, as appropriate) so that the changes take effect on the next reboot.
Method |
Description |
---|---|
SYSMAN utility |
Perform the following steps:
|
AUTOGEN utility |
Perform the following steps:
Do not specify the SHUTDOWN or REBOOT option. Hints: If your next action is to shut down the node, you can specify SHUTDOWN or REBOOT (in place of SETPARAMS) in the DO @SYS$UPDATE:AUTOGEN GETDATA command. |
Both of these methods propagate the values to the computer's ALPHAVMSSYS.PAR file on Alpha computers or to the IA64VMSSYS.PAR file on Integrity server systems. In order for these changes to take effect, continue with the instructions in either Section 8.6.2, “Shutting Down the Cluster” to shut down the cluster or in Section 8.6.3, “Shutting Down a Single Node” to shut down the node.
8.6.2. Shutting Down the Cluster
Log in to the system manager's account on any node in the cluster.
Run the SYSMAN utility and specify the SET ENVIRONMENT/CLUSTER command. Be sure to specify the /CLUSTER_SHUTDOWN qualifier to the SHUTDOWN NODE command. For example:
$
RUN SYS$SYSTEM:SYSMAN
SYSMAN>
SET ENVIRONMENT/CLUSTER
%SYSMAN-I-ENV, current command environment:
Clusterwide on local cluster
Username SYSTEM will be used on nonlocal nodes
SYSMAN>
SHUTDOWN NODE/CLUSTER_SHUTDOWN/MINUTES_TO_SHUTDOWN=5 -
_SYSMAN> /AUTOMATIC_REBOOT/REASON="Cluster Reconfiguration"
%SYSMAN-I-SHUTDOWN, SHUTDOWN request sent to node %SYSMAN-I-SHUTDOWN, SHUTDOWN request sent to node SYSMAN> SHUTDOWN message on BHAGAT from user SYSTEM at BHAGAT Batch 11:02:10 BHAGAT will shut down in 5 minutes; back up shortly via automatic reboot. Please log off node BHAGAT. Cluster Reconfiguration SHUTDOWN message on BHAGAT from user SYSTEM at BHAGAT Batch 11:02:10 PLUTO will shut down in 5 minutes; back up shortly via automatic reboot. Please log off node PLUTO. Cluster Reconfiguration
For more information, see Section 10.6, “Shutting Down a Cluster”.
8.6.3. Shutting Down a Single Node
Method |
Description |
---|---|
SYSMAN utility |
Follow these steps:
|
SHUTDOWN command procedure |
Follow these steps:
|
For more information, see Section 10.6, “Shutting Down a Cluster”.
8.6.4. Updating Network Data
Whenever you add a satellite, the cluster configuration command procedure you use (CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM) updates both the permanent and volatile remote node network databases (NETNODE_REMOTE.DAT) on the boot server. However, the volatile databases on other cluster members are not automatically updated.
$
RUN SYS$SYSTEM:SYSMAN
SYSMAN>
SET ENVIRONMENT/CLUSTER
%SYSMAN-I-ENV, current command environment: Clusterwide on lo