Cluster Systems

Operating System and Version:
VSI OpenVMS Alpha Version 8.4-2L1 or higher
VSI OpenVMS IA-64 Version 8.4-1H1 or higher
VSI OpenVMS x86-64 Version 9.2-1 or higher

Preface

This manual describes cluster concepts, procedures, and guidelines for configuring and managing OpenVMS Cluster systems. Except where noted, the procedures and guidelines apply equally to Integrity servers and Alpha systems. This manual also includes information for providing high availability, building-block growth, and unified system management across coupled systems.

1. About VSI

VMS Software, Inc. (VSI) is an independent software company licensed by Hewlett Packard Enterprise to develop and support the OpenVMS operating system.

2. Introduction

This guide describes system management for OpenVMS Cluster systems. Although the OpenVMS Cluster software for each of the supported architectures is separately purchased, licensed, and installed, the difference between the architectures lies mainly in the hardware used. Essentially, system management for Integrity servers, Alpha computers, and x86-64 computers in an OpenVMS Cluster is identical. Exceptions are pointed out.

3. Who Should Use This Manual

This document is intended for anyone responsible for setting up and managing OpenVMS Cluster systems. To use the document as a guide to cluster management, you must have a thorough understanding of system management concepts and procedures, as described in the VSI OpenVMS System Manager's Manual.

4. How This Manual Is Organized

VSI OpenVMS Cluster Systems contains of the following chapters and appendixes.

Chapter 1, Introduction to OpenVMS Cluster System Management introduces OpenVMS Cluster systems.

Chapter 2, OpenVMS Cluster Concepts presents the software concepts integral to maintaining OpenVMS Cluster membership and integrity.

Chapter 3, OpenVMS Cluster Interconnect Configurations describes various OpenVMS Cluster configurations and the ways they are interconnected.

Chapter 4, The OpenVMS Cluster Operating Environment explains how to set up an OpenVMS Cluster system and coordinate system files.

Chapter 5, Preparing a Shared Environment explains how to set up an environment in which resources can be shared across nodes in the OpenVMS Cluster system.

Chapter 6, Cluster Storage Devices discusses disk and tape management concepts and procedures and how to use Volume Shadowing for OpenVMS to prevent data unavailability.

Chapter 7, Setting Up and Managing Cluster Queues discusses queue management concepts and procedures.

Chapter 8, Configuring an OpenVMS Cluster System explains how to build an OpenVMS Cluster system once the necessary preparations are made, and how to reconfigure and maintain the cluster.

Chapter 9, Building Large OpenVMS Cluster Systems provides guidelines for configuring and building large OpenVMS Cluster systems, booting satellite nodes, and cross-architecture booting.

Chapter 10, Maintaining an OpenVMS Cluster System describes ongoing OpenVMS Cluster system maintenance.

Appendix A, Cluster System Parameters lists and defines OpenVMS Cluster system parameters.

Appendix B, Building Common Files provides guidelines for building a cluster common user authorization file.

Appendix C, Cluster Troubleshooting provides troubleshooting information.

Appendix D, Sample Programs for LAN Control presents three sample programs for LAN control and explains how to use the Local Area OpenVMS Cluster Network Failure Analysis Program.

Appendix E, Subroutines for LAN Control describes the subroutine package used with local area OpenVMS Cluster sample programs.

Appendix F, Troubleshooting the NISCA Protocol provides techniques for troubleshooting network problems related to the NISCA transport protocol.

Appendix G, NISCA Transport Protocol Congestion Control describes how the interactions of work load distribution and network topology affect OpenVMS Cluster system performance, and discusses transmit channel selection by PEDRIVER.

5. Related Documents

This document is not a one-volume reference manual. The utilities and commands are described in detail in the VSI OpenVMS System Manager's Manual, the VSI OpenVMS System Management Utilities Reference Manual, and the VSI OpenVMS DCL Dictionary.

For additional information on the topics covered in this manual, see the following documents:
  • Guidelines for OpenVMS Cluster Configurations

  • VSI OpenVMS Alpha Partitioning and Galaxy Guide

  • VSI OpenVMS Guide to OpenVMS File Applications

  • VSI OpenVMS Guide to System Security

  • OpenVMS Alpha System Dump Analyzer Utility Manual

  • VMS System Dump Analyzer Utility Manual

  • VSI OpenVMS I/O User's Reference Manual

  • VSI OpenVMS License Management Utility Guide

  • VSI OpenVMS System Management Utilities Reference Manual

  • VSI OpenVMS System Manager's Manual

  • A Comparison of System Management on OpenVMS AXP and OpenVMS VAX

  • VSI OpenVMS System Services Reference Manual

  • VSI OpenVMS Volume Shadowing Guide

  • VSI OpenVMS Software Product Descriptions (available on the VSI OpenVMS website: https://vmssoftware.com/resources/documentation/)

  • VSI OpenVMS DECnet Network Management Utilities

  • VSI OpenVMS DECnet Networking Manual

  • The VSI DECnet–Plus (formerly known as DECnet/OSI) documentation set

  • The TCP/IP Services for OpenVMS documentation set

6. VSI Encourages Your Comments

You may send comments or suggestions regarding this manual or any VSI document by sending electronic mail to the following Internet address: . Users who have VSI OpenVMS support contracts through VSI can contact for help with this product.

7. OpenVMS Documentation

The full VSI OpenVMS documentation set can be found on the VMS Software Documentation webpage at https://docs.vmssoftware.com.

8. Conventions

The following conventions are used in this manual:
ConventionMeaning

A horizontal ellipsis in examples indicates one of the following possibilities:
  • Additional optional arguments in a statement have been omitted.

  • The preceding item or items can be repeated one or more times.

  • Additional parameters, values, or other information can be entered.

A vertical ellipsis indicates the omission of items from a code example or command format; the items are omitted because they are not important to the topic being discussed.

( )

In command format descriptions, parentheses indicate that you must enclose choices in parentheses if you specify more than one.

[ ]

In command format descriptions, brackets indicate optional choices. You can choose one or more items or no items. Do not type the brackets on the command line. However, you must include the brackets in the syntax for OpenVMS directory specifications and for a substring specification in an assignment statement.

|

In command format descriptions, vertical bars separate choices within brackets or braces. Within brackets, the choices are optional; within braces, at least one choice is required. Do not type the vertical bars on the command line.

{ }

In command format descriptions, braces indicate required choices; you must choose at least one of the items listed. Do not type the braces on the command line.

bold text

This typeface represents the name of an argument, an attribute, or a reason.

italic text

Italic text indicates important information, complete titles of manuals, or variables. Variables include information that varies in system output (Internal error number), in command lines (/PRODUCER= name), and in command parameters in text (where dd represents the predefined code for the device type).

UPPERCASE TEXT

Uppercase text indicates a command, the name of a routine, the name of a file, or the abbreviation for a system privilege.

Monospace text

Monospace type indicates code examples and interactive screen displays.

In the C programming language, monospace type in text identifies the following elements:keywords, the names of independently compiled external functions and files, syntax summaries, and references to variables or identifiers introduced in an example.

-

A hyphen at the end of a command format description, command line, or code line indicates that the command or statement continues on the following line.

numbers

All numbers in text are assumed to be decimal unless otherwise noted. Nondecimal radixes—binary, octal, or hexadecimal—are explicitly indicated.

Chapter 1. Introduction to OpenVMS Cluster System Management

Cluster technology was pioneered by Digital Equipment Corporation in 1983 with the VAXcluster system. The VAXcluster system was built using multiple standard VAX computing systems and the VMS operating system. The initial VAXcluster system offered the power and manageability of a centralized system and the flexibility of many physically distributed computing systems.

Through the years, the technology has evolved to support mixed-architecture cluster systems and the name changed to OpenVMS Cluster systems. Initially, OpenVMS Alpha and OpenVMS VAX systems were supported in a mixed-architecture OpenVMS Cluster system. In OpenVMS Version 8.2, cluster support was introduced for the OpenVMS Integrity server systems either in a single architecture cluster or in a mixed-architecture cluster with OpenVMS Alpha systems. VSI continues to enhance and expand OpenVMS Cluster capabilities. In OpenVMS Version 9.x, cluster support for x86-64 systems was included, both in single or mixed-architecture cluster configurations.

1.1. Overview

An OpenVMS Cluster system is a highly integrated organization of OpenVMS software, Alpha, VAX, Integrity, or x86-64 servers (or a combination of them), and storage devices that operate as a single system. The OpenVMS Cluster acts as a single virtual system, even though it is made up of many distributed systems. As members of an OpenVMS Cluster system, server systems can share processing resources, data storage, and queues under a single security and management domain, yet they can boot or shut down independently.

The distance between the computers in an OpenVMS Cluster system depends on the interconnects that you use. The computers can be located in one computer lab, on two floors of a building, between buildings on a campus, or on two different sites hundreds of miles apart.

An OpenVMS Cluster system, with computers located on two or more sites, is known as a multiple-site OpenVMS Cluster system. For more information about multiple site clusters, see the Guidelines for OpenVMS Cluster Configurations.

1.1.1. Uses

OpenVMS Cluster systems are an ideal environment for developing high-availability applications, such as transaction processing systems, servers for network client or server applications, and data-sharing applications.

1.1.2. Benefits

Computers in an OpenVMS Cluster system interact to form a cooperative, distributed operating system and derive a number of benefits, as shown in the following table.

Benefit

Description

Resource sharing

OpenVMS Cluster software automatically synchronizes and load balances batch and print queues, storage devices, and other resources among all cluster members.

Flexibility

Application programmers do not have to change their application code, and users do not have to know anything about the OpenVMS Cluster environment to take advantage of common resources.

High availability

System designers can configure redundant hardware components to create highly available systems that eliminate or withstand single points of failure.

Nonstop processing

The OpenVMS operating system, which runs on each node in an OpenVMS Cluster, facilitates dynamic adjustments to changes in the configuration.

Scalability

Organizations can dynamically expand computing and storage resources as business needs grow or change without shutting down the system or applications running on the system.

Performance

An OpenVMS Cluster system can provide high performance.

Management

Rather than repeating the same system management operation on multiple OpenVMS systems, management tasks can be performed concurrently for one or more nodes.

Security

Computers in an OpenVMS Cluster share a single security database that can be accessed by all nodes in a cluster.

Load balancing

OpenVMS Cluster systems distribute work across cluster members based on the current load of each member.

1.2. Hardware Components

OpenVMS Cluster system configurations consist of hardware components from the following general groups:
  • Computers

  • Interconnects

  • Storage devices

References: Detailed OpenVMS Cluster configuration guidelines can be found in the VSI OpenVMS Cluster Software Product Description (SPD) and in Guidelines for OpenVMS Cluster Configurations.

1.2.1. Computers

Up to 96 computers, ranging from desktop to mainframe systems, can be members of an OpenVMS Cluster system. Active members that run the OpenVMS Alpha or OpenVMS Integrity server operating system and participate fully in OpenVMS Cluster negotiations can include:
  • Integrity server computers or workstations

  • Alpha computers or workstations

  • x86-64 computers or workstations

1.2.2. Physical Interconnects

An interconnect is a physical path that connects computers to other computers and to storage subsystems. OpenVMS Cluster systems support a variety of interconnects (also referred to as buses) so that members can communicate using the most appropriate and effective method possible:
  • LAN (Ethernet)

  • Internet Protocol (IP) (Ethernet)

  • MEMORY CHANNEL (node to node communications, Alpha only)

  • Serial Attached SCSI (SAS) (node-to-storage only, Integrity servers only)

  • Small Computer Systems Interface (SCSI) (node-to-storage only)

  • Fibre Channel (FC) (node-to-storage only)


Note

The CI, DSSI, and FDDI interconnects are supported on Alpha and VAX systems. Memory Channel interconnects are supported only on Alpha systems.


Table 1.1. Interconnect Support by OpenVMS Platform

Interconnect

Platform Support

Comments

IP: UDP

Integrity servers and Alpha

Supports Fast Ethernet/Gigabit Ethernet/10 Gb Ethernet. 10 Gb Ethernet is supported on Ethernet devices only.

Fibre Channel

Integrity servers and Alpha

Shared storage only

SAS

Integrity servers

SCSI

Integrity servers and Alpha

Limited shared storage configurations only

LAN: Ethernet

Integrity servers and Alpha

The connection speed depends on NICs supported on the system and the NIC speed setting.

MEMORY CHANNEL

Alpha

Node-to-node communications only

For the most recent list of supported interconnects and speeds, see the VSI OpenVMS Cluster Software Software Product Description.

1.2.3. OpenVMS Galaxy SMCI

In addition to the physical interconnects listed in Section 1.2.2, “Physical Interconnects”, another type of interconnect, a shared memory CI (SMCI) for OpenVMS Galaxy instances, is available. SMCI supports cluster communications between Galaxy instances.

For more information about SMCI and Galaxy configurations, see the VSI OpenVMS Alpha Partitioning and Galaxy Guide.

1.2.4. Storage Devices

A shared storage device is a disk or tape that is accessed by multiple computers in the cluster. Nodes access remote disks and tapes by means of the MSCP and TMSCP server software (described in Section 1.3.1, “OpenVMS Cluster Software Functions”).

Systems within an OpenVMS Cluster support a wide range of storage devices:
  • Disks and disk drives, including:
    • Fibre Channel (FC) disks

    • SAS devices

    • SCSI devices

    • Embedded devices, such as IDE and USB devices

    • Digital Storage Architecture (DSA) disks

    • RF series integrated storage elements (ISEs)

    • Solid state disks

  • Tapes and tape drives

  • Controllers and I/O servers, including the following:

    Controller

    Interconnect

    HSG and HSV

    FC

    LSI 1068 and LSI Logic 1068e

    SAS

    HSZ

    SCSI

    In addition, the K.scsi HSC controller allows the connection of the StorageWorks arrays with SCSI devices on the HSC storage subsystems.

    Note: HSZ controllers support many combinations of SDIs (standard disk interfaces) and STIs (standard tape interfaces) that connect disks and tapes.

For the most recent list of supported storage devices, see the VSI OpenVMS Version 8.4 Software Product Description.

In case Alpha Server Supported Options Lists or Integrity servers Supported Options Lists are needed, please contact to the HPE Support: https://support.hpe.com/.

1.3. Software Components

The OpenVMS operating system, which runs on each node in an OpenVMS Cluster, includes several software components that facilitate resource sharing and dynamic adjustments to changes in the underlying hardware configuration.

If one computer becomes unavailable, the OpenVMS Cluster system continues operating because OpenVMS is still running on the remaining computers.

1.3.1. OpenVMS Cluster Software Functions

The following table describes the software components and their main function.
ComponentFacilitatesFunction

Connection manager

Member integrity

Coordinates participation of computers in the cluster and maintains cluster integrity when computers join or leave the cluster.

Distributed lock manager

Resource synchronization

Synchronizes operations of the distributed file system, job controller, device allocation, and other cluster facilities. If an OpenVMS Cluster computer shuts down, all locks that it holds are released so that processing can continue on the remaining computers.

Distributed file system

Resource sharing

Allows all computers to share access to mass storage and file records, regardless of the type of storage device (DSA, RF, SCSI, and solid state subsystem) or its location.

Distributed job controller

Queuing

Makes generic and execution queues available across the cluster.

MSCP server

Disk serving

Implements the proprietary mass storage control protocol in order to make disks available to all nodes that do not have direct access to those disks.

TMSCP server

Tape serving

Implements the proprietary tape mass storage control protocol in order to make tape drives available to all nodes that do not have direct access to those tape drives.

1.4. Communications

The System Communications Architecture (SCA) defines the communications mechanisms that allow nodes in an OpenVMS Cluster system to co-operate. SCA governs the sharing of data between resources at the nodes and binds together System Applications (SYSAPs) that run on different Integrity server systems and Alpha computers.

SCA consists of the following hierarchy of components:
Communications SoftwareFunction

System applications (SYSAPs)

Consists of clusterwide applications (for example, disk and tape class drivers, connection manager, and MSCP server) that use SCS software for interprocessor communication.

System Communications Services (SCS)

Provides basic connection management and communication services, implemented as a logical path between system applications (SYSAPs) on nodes in an OpenVMS Cluster system.

Port drivers

Control the communication paths between local and remote ports.

Physical interconnects

Consists of ports or adapters for CI, DSSI, Ethernet, ATM, FDDI, and MEMORY CHANNEL interconnects. PEDRIVER is the port driver for LAN (Ethernet) interconnect and starting with OpenVMS Version 8.4 PEDRIVER is also enabled to use TCP/IP for cluster communication.

1.4.1. System Communications

Figure 1.1, “OpenVMS Cluster System Communications” shows the relationship between OpenVMS Cluster components.

Figure 1.1. OpenVMS Cluster System Communications
OpenVMS Cluster System Communications
In Figure 1.1, “OpenVMS Cluster System Communications”, processes in different nodes exchange information with each other.
  • Processes can call the $QIO system service and other system services directly from a program or indirectly using other mechanisms such as OpenVMS Record Management Services (RMS). The $QIO system service initiates all I/O requests.

  • A SYSAP on one OpenVMS Cluster node communicates with a SYSAP on another node using a logical path called connection. For example, a connection manager on one node communicates with the connection manager on another node, or a disk class driver on one node communicates with the MSCP server on another node. The following SYSAPs use SCS for cluster communication:
    • Disk and tape class drivers

    • MSCP server

    • TMSCP server

    • DECnet class driver

    • Connection manager

    • SCA$TRANSPORT, which provides node-to-node communications to the intracluster communication (ICC) programming interface, available through ICC system services, and to the distributed queue manager

  • SCS routines provide connection setup and services to format and transfer SYSAP messages to a port driver for delivery over a specific interconnect.

  • Communications go through the port drivers to port drivers on other OpenVMS Cluster computers and storage controllers. A port driver manages a logical path, called a virtual circuit, between each pair of ports in an OpenVMS Cluster system. A virtual circuit provides reliable message delivery for the connections multiplexed upon it.

    Starting with OpenVMS Version 8.4, cluster systems can use Transmission Control Protocol and Internet Protocol (TCP/IP) stack for cluster communication. PEDRIVER is enhanced with the capability to use TCP/IP in addition to LAN for cluster communication. For more information, see Chapter 3, OpenVMS Cluster Interconnect Configurations.

1.4.2. Application Communications

Applications running on OpenVMS Cluster systems use TCP/IP, DECnet, or ICC for application communication.

ICC allows processes to efficiently exchange messages with processes running on other cluster members using system communications services and the underlying cluster interconnect. The DECnet and TCP/IP communication services allow processes to locate or start remote servers and then exchange messages.

Note

The generic references to DECnet in this document mean either DECnet for OpenVMS or DECnet-Plus (formerly known as DECnet/OSI) software.

1.4.3. Cluster Alias

DECnet provides a feature known as a cluster alias. A cluster alias is a collective name for the nodes in an OpenVMS Cluster system.

Application software can use the cluster alias as the name to connect to a node in the OpenVMS Cluster. DECnet chooses the node to which the application makes a connection. The use of a cluster alias frees the application from keeping track of individual nodes in the OpenVMS Cluster system and results in design simplification, configuration flexibility, and application availability. It also provides a mechanism for load balancing by distributing incoming connections across the nodes comprising the cluster.

1.4.4. failSAFE IP

TCP/IP provides a feature known as a failSAFE IP that allows IP addresses to failover when interfaces cease functioning on a system, where multiple interfaces have been configured with the same IP address.

You can configure a standby failover target IP address that failSAFE IP assigns to multiple interfaces on a node or across the OpenVMS Cluster system. When, for example, a Network Interface Controller fails or a cable breaks or disconnects, failSAFE IP activates the standby IP address so that an alternate interface can take over to maintain the network connection. If an address is not preconfigured with a standby, then failSAFE IP removes the address from the failed interface until it recovers. When the failed interface recovers, failSAFE IP detects this and can return its IP address.

1.5. System Management

The OpenVMS Cluster system manager must manage multiple users and resources for maximum productivity and efficiency while maintaining the necessary security.

1.5.1. Ease of Management

An OpenVMS Cluster system is easily managed because the multiple members, hardware, and software are designed to co-operate as a single system:
  • Smaller configurations usually include only one system disk (or two for an OpenVMS Cluster configuration with both OpenVMS Alpha and OpenVMS Integrity server operating systems), regardless of the number or location of computers in the configuration.

  • Software must be installed only once for each operating system (Alpha or Integrity servers), and is accessible by every user and node of the OpenVMS Cluster.

  • Users must be added once to access the resources of the entire OpenVMS Cluster.

  • Several system management utilities and commands facilitate cluster management.

Figure 1.2, “Single-Point OpenVMS Cluster System Management” illustrates centralized system management.

Figure 1.2. Single-Point OpenVMS Cluster System Management
Single-Point OpenVMS Cluster System Management

1.5.2. Tools and Utilities

The OpenVMS operating system supports a number of utilities and tools to assist you with the management of the distributed resources in OpenVMS Cluster configurations. Proper management is essential to ensure the availability and performance of OpenVMS Cluster configurations.

OpenVMS and its partners offer a wide selection of tools to meet diverse system management needs. Table 1.2, “System Management Tools” describes the products available for cluster management and indicates whether each is supplied with the operating system or is an optional product, which is purchased separately.
Table 1.2. System Management Tools
ToolSupplied or OptionalFunction

Accounting

VMS Accounting

Supplied

Tracks how resources are being used.

Configuration and capacity planning

LMF (License Management Facility)

Supplied

Helps the system manager determine which software products are licensed and installed on a standalone system and on each computer in an OpenVMS Cluster system.

SYSGEN (System Generation) utility

Supplied

Allows you to tailor your system for a specific hardware and software configuration. Use SYSGEN to modify system parameters, load device drivers, and create additional page and swap files.

CLUSTER_CONFIG.COM

Supplied

Automates the configuration or reconfiguration of an OpenVMS Cluster system and assumes the use of DECnet (for satellite booting).

CLUSTER_CONFIG_LAN.COM

Supplied

Automates configuration or reconfiguration of an OpenVMS Cluster system using LANCP/LANACP (for satellite booting).

HPE Management Agents for OpenVMS

Supplied

Consists of a web server for system management with management agents that allow you to look at devices on your OpenVMS systems.

HPE Insight Manager XE

Supplied with every HPNT server

Centralizes system management in one system to reduce cost, improve operational efficiency and effectiveness, and minimize system down time. You can use HPE Insight Manager XE on an NT server to monitor every system in an OpenVMS Cluster system. In a configuration of heterogeneous VSI systems, you can use HPE Insight Manager XE on an NT server to monitor all systems.

Event and fault tolerance

OPCOM message routing

Supplied

Provides event notification.

Operations management

Clusterwide process services

Supplied

Allows OpenVMS system management commands, such as SHOW USERS, SHOW SYSTEM, and STOP/ID=, to operate clusterwide.

Availability Manager

Supplied

From either an OpenVMS system or a Windows node, enables you to monitor one or more OpenVMS nodes on an extended LAN or wide area network (WAN). That is, the nodes for which you are collecting the information must be in the same extended LAN and there should be an interface that communicates with the collector nodes as well as the WAN analyzer. The Availability Manager collects system and process data from multiple OpenVMS nodes simultaneously, and then analyzes the data and displays the output using a native Java GUI.

HPE WBEM Services for OpenVMS

Supplied

WBEM (Web-Based Enterprise Management) enables management applications to retrieve system information and request system operations wherever and whenever required. It allows customers to manage their systems consistently across multiple platforms and operating systems, providing integrated solutions that optimize your infrastructure for greater operational efficiency.

SCACP (Systems Communications Architecture Control Program)

Supplied

Enables you to monitor, manage, and diagnose cluster communications and cluster interconnects.

DNS (Distributed Name Service)

Optional

Configures certain network nodes as name servers that associate objects with network names.

LATCP (Local Area Transport Control Program)

Supplied

Provides the function to control and obtain information from LAT port driver.

LANCP (LAN Control Program)

Supplied

Allows the system manager to configure and control the LAN software on OpenVMS systems.

NCP (Network Control Protocol) utility

Optional

Allows the system manager to supply and access information about the DECnet for OpenVMS (Phase IV) network from a configuration database.

NCL (Network Control Language) utility

Optional

Allows the system manager to supply and access information about the DECnet–Plus network from a configuration database.

POLYCENTER Software Installation Utility (PCSI)

Supplied

Provides rapid installations of software products.

Queue Manager

Supplied

Uses OpenVMS Cluster generic and execution queues to feed node-specific queues across the cluster.

Show Cluster utility

Supplied

Monitors activity and performance in an OpenVMS Cluster configuration, then collects and sends information about that activity to a terminal or other output device.

SDA (System Dump Analyzer)

Supplied

Allows you to inspect the contents of memory saved in the dump taken at crash time or as it exists in a running system. You can use SDA interactively or in batch mode.

SYSMAN (System Management utility)

Supplied

Enables device and processor control commands to take effect across an OpenVMS Cluster.

VMSINSTAL

Supplied

Provides software installations.

Performance

AUTOGEN utility

Supplied

Optimizes system parameter settings based on usage.

Monitor utility

Supplied

Provides basic performance data.

Security

Authorize utility

Supplied

Modifies user account profiles.

SET ACL command

Supplied

Sets complex protection on many system objects.

SET AUDIT command

Supplied

Facilitates tracking of sensitive system objects.

Storage management

Backup utility

Supplied

Allows OpenVMS Cluster system managers to create backup copies of files and directories from storage media and then restore them. This utility can be used on one node to back up data stored on disks throughout the OpenVMS Cluster system.

Mount utility

Supplied

Enables a disk or tape volume for processing by one computer, a subset of OpenVMS Cluster computers, or all OpenVMS Cluster computers.

Volume Shadowing for OpenVMS

Optional

Replicates disk data across multiple disks to help OpenVMS Cluster systems survive disk failures.

1.5.3. System Management Tools from OpenVMS Partners

OpenVMS Partners offer a wide selection of tools to meet diverse system management needs. The types of tools are described in the following list:
  • Schedule managers

    Enable specific actions to be triggered at determined times, including repetitive and periodic activities, such as nightly backups.

  • Event managers

    Monitor a system and report occurrences and events that may require an action or that may indicate a critical or alarming situation, such as low memory or an attempted security breaking.

  • Console managers

    Enable a remote connection to and emulation of a system console so that system messages can be displayed and commands can be issued.

  • Performance managers

    Monitor system performance by collecting and analyzing data to allow proper tailoring and configuration of system resources. Performance managers might also collect historical data for capacity planning.

For information about OpenVMS Partners and the tools they provide, contact to the HPE Support: https://support.hpe.com/

1.5.4. Other Configuration Aids

In addition to these utilities and partner products, several commands are available that allow the system manager to set parameters on Fibre Channel, SCSI and SAS storage subsystems to help configure and manage the system. See the appropriate hardware documentation for more information.

Chapter 2. OpenVMS Cluster Concepts

To help you understand the design and implementation of an OpenVMS Cluster system, this chapter describes its basic architecture.

2.1. OpenVMS Cluster System Architecture

Figure 2.1, “OpenVMS Cluster System Architecture” illustrates the protocol layers within the OpenVMS Cluster system architecture, ranging from the communications mechanisms at the base of the figure to the users of the system at the top of the figure. These protocol layers include:
  • Ports

  • System Communications Services (SCS)

  • System Applications (SYSAPs)

  • Other layered components

Figure 2.1. OpenVMS Cluster System Architecture
OpenVMS Cluster System Architecture

Note

Not all interconnects are supported on all three architectures of OpenVMS. The CI, DSSI, and FDDI interconnects are supported on Alpha and VAX systems. Memory Channel and ATM interconnects are supported only on Alpha systems.

2.1.1. Port Layer

This lowest level of the architecture provides connections, in the form of communication ports and physical paths, between devices. The port layer can contain any of the following interconnects:
  • LAN, Ethernet

  • Internet Protocol (IP), Ethernet

  • MEMORY CHANNEL

  • SAS

  • SCSI

  • Fibre Channel

Each interconnect is accessed by a port (also referred to as an adapter) that connects to the processor node. For example, the Fibre Channel interconnect is accessed by way of a Fibre Channel port.

2.1.2. SCS Layer

The SCS layer provides basic connection management and communications services in the form of datagrams, messages, and block transfers over each logical path. Table 2.1, “Communications Services” describes these services.
Table 2.1. Communications Services

Service

Delivery Guarantees

Usage

Datagrams

Information units that fit in 1 packet or less.

Delivery of datagrams is not guaranteed. Datagrams can be lost, duplicated, or delivered out of order.

Status and information messages whose loss is not critical.

Applications that have their own reliability protocols such as DECnet or TCP/IP.

Messages

Information units that fit in 1 packet or less.

Messages are guaranteed to be delivered and to arrive in order. Virtual circuit sequence numbers are used on the individual packets.

Disk read and write requests.

Block data transfers

Copying (that is, reading or writing) any contiguous data between a local process or system virtual address space and an address on another node. Individual transfers are limited to the lesser of 232-1 bytes, or the physical memory constraints of the host. Block data is a form of remote DMA transfer.

Delivery of block data is guaranteed. The sending and receiving ports and the port emulators cooperate in breaking the transfer into data packets and ensuring that all packets are correctly transmitted, received, and placed in the appropriate destination buffer. Block data transfers differ from messages in the size of the transfer.

Disk subsystems and disk servers to move data associated with disk read and write requests. Fast remastering of large lock trees. Transferring large ICC messages.

The SCS layer is implemented as a combination of hardware and software, or software only, depending upon the type of port. SCS manages connections in an OpenVMS Cluster and multiplexes messages between system applications over a common transport called a virtual circuit. A virtual circuit exists between each pair of SCS ports and a set of SCS connections that are multiplexed on that virtual circuit.

2.1.3. System Applications (SYSAPs) Layer

The next higher layer in the OpenVMS Cluster architecture consists of the SYSAPs layer. This layer consists of multiple system applications that provide, for example, access to disks and tapes and cluster membership control. SYSAPs can include:
  • Connection manager

  • MSCP server

  • TMSCP server

  • Disk and tape class drivers

These components are described in detail later in this chapter.

2.1.4. Other Layered Components

A wide range of OpenVMS components layer on top of the OpenVMS Cluster system architecture, including:
  • Volume Shadowing for OpenVMS

  • Distributed lock manager

  • Process control services

  • Distributed file system

  • Record Management Services (RMS)

  • Distributed job controller

These components, except for volume shadowing, are described in detail later in this chapter. Volume Shadowing for OpenVMS is described in Section 6.6, “Shadowing Disks Across an OpenVMS Cluster”.

2.2. OpenVMS Cluster Software Functions

The OpenVMS Cluster software components that implement OpenVMS Cluster communication and resource-sharing functions always run on every computer in the OpenVMS Cluster. If one computer fails, the OpenVMS Cluster system continues operating, because the components still run on the remaining computers.

2.2.1. Functions

The following table summarizes the OpenVMS Cluster communication and resource-sharing functions and the components that perform them.

Function

Performed By

Ensure that OpenVMS Cluster computers communicate with one another to enforce the rules of cluster membership

Connection manager

Synchronize functions performed by other OpenVMS Cluster components, OpenVMS products, and other software components

Distributed lock manager

Share disks and files

Distributed file system

Make disks available to nodes that do not have direct access

MSCP server

Make tapes available to nodes that do not have direct access

TMSCP server

Make queues available

Distributed job controller

2.3. Ensuring the Integrity of Cluster Membership

The connection manager ensures that computers in an OpenVMS Cluster system communicate with one another to enforce the rules of cluster membership.

Computers in an OpenVMS Cluster system share various data and system resources, such as access to disks and files. To achieve the coordination that is necessary to maintain resource integrity, the computers must maintain a clear record of cluster membership.

2.3.1. Connection Manager

The connection manager creates an OpenVMS Cluster when the first computer is booted and reconfigures the cluster when computers join or leave it during cluster state transitions. The overall responsibilities of the connection manager are to:
  • Prevent partitioning (see Section 2.3.2, “Cluster Partitioning”).

  • Track which nodes in the OpenVMS Cluster system are active and which are not.

  • Deliver messages to remote nodes.

  • Remove nodes.

  • Provide a highly available message service in which other software components, such as the distributed lock manager, can synchronize access to shared resources.

2.3.2. Cluster Partitioning

A primary purpose of the connection manager is to prevent cluster partitioning, a condition in which nodes in an existing OpenVMS Cluster configuration divide into two or more independent clusters.

Cluster partitioning can result in data file corruption because the distributed lock manager cannot coordinate access to shared resources for multiple OpenVMS Cluster systems. The connection manager prevents cluster partitioning using a quorum algorithm.

2.3.3. Quorum Algorithm

The quorum algorithm is a mathematical method for determining if a majority of OpenVMS Cluster members exist so that resources can be shared across an OpenVMS Cluster system. Quorum is the number of votes that must be present for the cluster to function. Quorum is a dynamic value calculated by the connection manager to prevent cluster partitioning. The connection manager allows processing to occur only if a majority of the OpenVMS Cluster members are functioning.

2.3.4. System Parameters

Two system parameters, VOTES and EXPECTED_VOTES, are key to the computations performed by the quorum algorithm. The following table describes these parameters.
ParameterDescription

VOTES

Specifies a fixed number of votes that a computer contributes toward quorum. The system manager can set the VOTES parameters on each computer or allow the operating system to set it to the following default values:
  • For satellite nodes, the default value is 0.

  • For all other computers, the default value is 1.

Each Integrity server or an Alpha computer with a nonzero value for the VOTES system parameter is considered a voting member.

EXPECTED_VOTES

Specifies the sum of all VOTES held by OpenVMS Cluster members. The initial value is used to derive an estimate of the correct quorum value for the cluster. The system manager must set this parameter on each active Integrity server system or an Alpha system, including satellites in the cluster.

2.3.5. Calculating Cluster Votes

The quorum algorithm operates as follows:

Step

Action

1

When nodes in the OpenVMS Cluster boot, the connection manager uses the largest value for EXPECTED_VOTES of all systems present to derive an estimated quorum value according to the following formula:
Estimated quorum = (EXPECTED_VOTES + 2)/2 | Rounded down

2

During a state transition (whenever a node enters or leaves the cluster or when a quorum disk is recognized),the connection manager dynamically computes the cluster quorum value to be the maximum of the following:
  • The current cluster quorum value (calculated during the last cluster transition).

  • Estimated quorum, as described in step 1.

  • The value calculated from the following formula, where the VOTES system parameter is the total votes held by all cluster members:
    QUORUM = (VOTES + 2)/2 | Rounded down

Note: Quorum disks are discussed in Section 2.3.8, “Quorum Disk”.

3

The connection manager compares the cluster votes value to the cluster quorum value and determines what action to take based on the following conditions:

WHEN...

THEN...

The total number of cluster votes is equal to at least the quorum value

The OpenVMS Cluster system continues running.

The current number of cluster votes drops below the quorum value (because of computers leaving the cluster)

The remaining OpenVMS Cluster members suspend all process activity and all I/O operations to cluster-accessible disks and tapes until sufficient votes are added (that is, enough computers have joined the OpenVMS Cluster) to bring the total number of votes to a value greater than or equal to quorum.

Note: When a node leaves the OpenVMS Cluster system, the connection manager does not decrease the cluster quorum value. In fact, the connection manager never decreases the cluster quorum value; the connection manager only increases the value, unless the REMOVE NODE option was selected during shutdown. However, system managers can decrease the value according to the instructions in Section 10.11.2, “Reducing Cluster Quorum Value”.

2.3.6. Example

Consider a cluster consisting of three computers, each computer having its VOTES parameter set to 1 and its EXPECTED_VOTES parameter set to 3. The connection manager dynamically computes the cluster quorum value to be 2 (that is, (3 + 2)/2). In this example, any two of the three computers constitute a quorum and can run in the absence of the third computer. No single computer can constitute a quorum by itself. Therefore, there is no way the three OpenVMS Cluster computers can be partitioned and run as two independent clusters.

2.3.7. Sub-Cluster Selection

To select the optimal sub-cluster and to continue after the communication failure occurs, two possible sub-clusters are compared as follows:
  1. The subset with the highest number of votes wins, if one of the subset has more votes.

  2. If in case there is a tie in the number of votes:
    • The subset with the higher number of nodes wins.

    • If the number of nodes is also tied, then: OpenVMS arbitrarily, but deterministically selects one of the two subsets to "win" based on a comparison of SCS System ID values.

2.3.8. Quorum Disk

A cluster system manager can designate a disk a quorum disk. The quorum disk acts as a virtual cluster member whose purpose is to add one vote to the total cluster votes. By establishing a quorum disk, you can increase the availability of a two-node cluster; such configurations can maintain quorum in the event of failure of either the quorum disk or one node, and continue operating.

Note: Setting up a quorum disk is recommended only for OpenVMS Cluster configurations with two nodes. A quorum disk is neither necessary nor recommended for configurations with more than two nodes.

For example, assume an OpenVMS Cluster configuration with many satellites (that have no votes) and two nonsatellite systems (each having one vote) that downline load the satellites. Quorum is calculated as follows:
(EXPECTED VOTES + 2)/2 = (2 + 2)/2 = 2

Because there is no quorum disk, if either nonsatellite system departs from the cluster, only one vote remains and cluster quorum is lost. Activity will be blocked throughout the cluster until quorum is restored.

However, if the configuration includes a quorum disk (adding one vote to the total cluster votes), and the EXPECTED_VOTES parameter is set to 3 on each node, then quorum will still be 2 even if one of the nodes leaves the cluster. Quorum is calculated as follows:
(EXPECTED VOTES + 2)/2 = (3 + 2)/2 = 2
Rules: Each OpenVMS Cluster system can include only one quorum disk. At least one computer must have a direct (not served) connection to the quorum disk:
  • Any computers that have a direct, active connection to the quorum disk or that have the potential for a direct connection should be enabled as quorum disk watchers.

  • Computers that cannot access the disk directly must rely on the quorum disk watchers for information about the status of votes contributed by the quorum disk.

Reference: For more information about enabling a quorum disk, see Section 8.2.4, “Adding a Quorum Disk”. Section 8.3.2, “Removing a Quorum Disk” describes removing a quorum disk.

2.3.9. Quorum Disk Watcher

To enable a computer as a quorum disk watcher, use one of the following methods:

Method

Perform These Steps

Run the CLUSTER_CONFIG.COM procedure (described in Chapter 8, Configuring an OpenVMS Cluster System)

Invoke the procedure and:
  1. Select the CHANGE option.

  2. From the CHANGE menu, select the item labeled Enable a quorum disk on the local computer.

  3. At the prompt, supply the quorum disk device name.

The procedure uses the information you provide to update the values of the DISK_QUORUM and QDSKVOTES system parameters.

Respond YES when the OpenVMS installation procedure asks whether the cluster will contain a quorum disk (described in Chapter 4, The OpenVMS Cluster Operating Environment)

During the installation procedure:
  1. Answer Y when the procedure asks whether the cluster will contain a quorum disk.

  2. At the prompt, supply the quorum disk device name.

The procedure uses the information you provide to update the values of the DISK_QUORUM and QDSKVOTES system parameters.

Edit the MODPARAMS or AGEN$ files (described in Chapter 8, Configuring an OpenVMS Cluster System)

Edit the following parameters:
  • DISK_QUORUM: Specify the quorum disk name, in ASCII, as a value for the DISK_QUORUM system parameter.

  • QDSKVOTES: Set an appropriate value for the QDSKVOTES parameter. This parameter specifies the number of votes contributed to the cluster votes total by a quorum disk. The number of votes contributed by the quorum disk is equal to the smallest value of the QDSKVOTES parameter on any quorum disk watcher.

Hint: If only one quorum disk watcher has direct access to the quorum disk, then remove the disk and give its votes to the node.

2.3.10. Rules for Specifying Quorum

For the quorum disk's votes to be counted in the total cluster votes, the following conditions must be met:
  • On all computers capable of becoming watchers, you must specify the same physical device name as a value for the DISK_QUORUM system parameter. The remaining computers (which must have a blank value for DISK_QUORUM) recognize the name specified by the first quorum disk watcher with which they communicate.

  • At least one quorum disk watcher must have a direct, active connection to the quorum disk.

  • The disk must contain a valid format file named QUORUM.DAT in the master file directory. The QUORUM.DAT file is created automatically after a system specifying a quorum disk has booted into the cluster for the first time. This file is used on subsequent reboots.

    Note: The file is not created if the system parameter STARTUP_P1 is set to MIN.

  • To permit recovery from failure conditions, the quorum disk must be mounted by all disk watchers.

  • The OpenVMS Cluster can include only one quorum disk.

  • The quorum disk cannot be a member of a shadow set.

Hint: By increasing the quorum disk's votes to one less than the total votes from both systems (and by increasing the value of the EXPECTED_VOTES system parameter by the same amount), you can boot and run the cluster with only one node.

2.4. State Transitions

OpenVMS Cluster state transitions occur when a computer joins or leaves an OpenVMS Cluster system and when the cluster recognizes a quorum disk state change. The connection manager controls these events to ensure the preservation of data integrity throughout the cluster.

A state transition's duration and effect on users (applications) are determined by the reason for the transition, the configuration, and the applications in use.

2.4.1. Adding a Member

Every transition goes through one or more phases, depending on whether its cause is the addition of a new OpenVMS Cluster member or the failure of a current member.

Table 2.2, “Transitions Caused by Adding a Cluster Member” describes the phases of a transition caused by the addition of a new member.
Table 2.2. Transitions Caused by Adding a Cluster Member

Phase

Description

New member detection

Early in its boot sequence, a computer seeking membership in an OpenVMS Cluster system sends messages to current members asking to join the cluster. The first cluster member that receives the membership request acts as the new computer's advocate and proposes reconfiguring the cluster to include the computer in the cluster. While the new computer is booting, no applications are affected.

Note: The connection manager will not allow a computer to join the OpenVMS Cluster system if the node's value for EXPECTED_VOTES would readjust quorum higher than calculated votes to cause the OpenVMS Cluster to suspend activity.

Reconfiguration

During a configuration change due to a computer being added to an OpenVMS Cluster, all current OpenVMS Cluster members must establish communications with the new computer. Once communications are established, the new computer is admitted to the cluster. In some cases, the lock database is rebuilt.

2.4.2. Losing a Member

Table 2.3, “Transitions Caused by Loss of a Cluster Member” describes the phases of a transition caused by the failure of a current OpenVMS Cluster member.
Table 2.3. Transitions Caused by Loss of a Cluster Member

Cause

Description

Failure detection

The duration of this phase depends on the cause of the failure and on how the failure is detected.

During normal cluster operation, messages sent from one computer to another are acknowledged when received.

IF...

THEN...

A message is not acknowledged within a period determined by OpenVMS Cluster communications software

The repair attempt phase begins.

A cluster member is shut down or fails

The operating system causes datagrams to be sent from the computer shutting down to the other members. These datagrams state the computer's intention to sever communications and to stop sharing resources. The failure detection and repair attempt phases are bypassed, and the reconfiguration phase begins immediately.

Repair attempt

If the virtual circuit to an OpenVMS Cluster member is broken, attempts are made to repair the path. Repair attempts continue for an interval specified by the PAPOLLINTERVAL system parameter. (System managers can adjust the value of this parameter to suit local conditions.) Thereafter, the path is considered irrevocably broken, and steps must be taken to reconfigure the OpenVMS Cluster system so that all computers can once again communicate with each other and so that computers that cannot communicate are removed from the OpenVMS Cluster.

Reconfiguration

If a cluster member is shut down or fails, the cluster must be reconfigured. One of the remaining computers acts as coordinator and exchanges messages with all other cluster members to determine an optimal cluster configuration with the most members and the most votes. This phase, during which all user (application) activity is blocked, usually lasts less than 3 seconds, although the actual time depends on the configuration.

OpenVMS Cluster system recovery

Recovery includes the following stages, some of which can take place in parallel:

Stage

Action

I/O completion

When a computer is removed from the cluster, OpenVMS Cluster software ensures that all I/O operations that are started prior to the transition complete before I/O operations that are generated after the transition. This stage usually has little or no effect on applications.

Recovery includes the following stages, some of which can take place in parallel:

When a computer is removed from the cluster, OpenVMS Cluster software ensures that all I/O operations that are started prior to the transition complete before I/O operations that are generated after the transition. This stage usually has little or no effect on applications.

WHEN...

THEN...

A computer leaves the OpenVMS Cluster

A rebuild is always performed.

A computer is added to the OpenVMS Cluster

A rebuild is performed when the LOCKDIRWT system parameter is greater than 1.

Caution:Setting the LOCKDIRWT system parameter to different values on the same model or type of computer can cause the distributed lock manager to use the computer with the higher value. This could cause undue resource usage on that computer.

Disk mount verification

This stage occurs only when the failure of a voting member causes quorum to be lost. To protect data integrity, all I/O activity is blocked until quorum is regained. Mount verification is the mechanism used to block I/O during this phase.

Quorum disk votes validation

If, when a computer is removed, the remaining members can determine that it has shut down or failed, the votes contributed by the quorum disk are included without delay in quorum calculations that are performed by the remaining members. However, if the quorum watcher cannot determine that the computer has shut down or failed (for example, if a console halt, power failure, or communications failure has occurred), the votes are not included for a period (in seconds) equal to four times the value of the QDSKINTERVAL system parameter. This period is sufficient to determine that the failed computer is no longer using the quorum disk.

Disk rebuild

If the transition is the result of a computer rebooting after a failure, the disks are marked as improperly dismounted.

Reference: See Sections 6.5.5 and 6.5.6 for information about rebuilding disks.

XFC cache change

If the XFC cache is active on this node, a check is made to determine if there are any nodes in the cluster that do not support the XFC cache. If so, any XFC cache data must be flushed before continuing with the cluster transition.

Clusterwide logical name recovery

This stage ensures that all nodes in the cluster have matching clusterwide logical name information.

Application recovery

When you assess the effect of a state transition on application users, consider that the application recovery phase includes activities such as replaying a journal file, cleaning up recovery units, and users logging in again.

2.5. OpenVMS Cluster Membership

OpenVMS Cluster systems based on LAN or IP network use a cluster group number and a cluster password to allow multiple independent OpenVMS Cluster systems to coexist on the same extended LAN or IP network and to prevent accidental access to a cluster by unauthorized computers.

Note

When using IP network for cluster communication, the remote node's IP address must be present in the SYS$SYSTEM:PE$IP_CONFIG.DAT local file.

2.5.1. Cluster Group Number

The cluster group number uniquely identifies each OpenVMS Cluster system on a LAN or IP or communicates by a common memory region (that is, communicating using SMCI). This group number must be either from 1 to 4095 or from 61440 to 65535.

Rule: If you plan to have more than one OpenVMS Cluster system on a LAN or an IP network, you must coordinate the assignment of cluster group numbers among system managers.

2.5.2. Cluster Password

The cluster password prevents an unauthorized computer using the cluster group number, from joining the cluster. The password must be from 1 to 31 characters; valid characters are letters, numbers, the dollar sign ($), and the underscore (_).

2.5.3. Location

The cluster group number and cluster password are maintained in the cluster authorization file, SYS$COMMON:[SYSEXE]CLUSTER_AUTHORIZE.DAT. This file is created during the installation of the operating system, if you indicate that you want to set up a cluster that utilizes the shared memory or the LAN. The installation procedure then prompts you for the cluster group number and password.

Note

If you convert an OpenVMS Cluster that uses only the CI or DSSI interconnect to one that includes a LAN or shared memory interconnect, the SYS$COMMON:[SYSEXE]CLUSTER_AUTHORIZE.DAT file is created when you execute the CLUSTER_CONFIG.COM command procedure, as described in Chapter 8, Configuring an OpenVMS Cluster System.

Reference: For information about OpenVMS Cluster group data in the CLUSTER_AUTHORIZE.DAT file, see Sections 8.4 and 10.8.

2.5.4. Example

If all nodes in the OpenVMS Cluster do not have the same cluster password, an error report similar to the following is logged in the error log file.
**** V3.4  ********************* ENTRY  343 ******************************** 
 
 
Logging OS                        1. OpenVMS 
System Architecture               2. Alpha 
OS version                           XC56-BL2 
Event sequence number           102. 
Timestamp of occurrence              16-SEP-2009 16:47:48 
Time since reboot                    0 Day(s) 1:04:52 
Host name                            PERK 
 
System Model                         AlphaServer ES45 Model 2 
 
Entry Type                       98. Asynchronous Device Attention 
 
 
---- Device Profile ---- 
Unit                                 PERK$PEA0 
Product Name                         NI-SCA Port 
 
---- NISCA Port Data ---- 
Error Type and SubType        x0600  Channel Error, Invalid Cluster Password 
                                     Received 
Status                    x0000000000000000 
Datalink Device Name                 EIA8: 
Remote Node Name                     CHBOSE 
Remote Address            x000064A9000400AA 
Local Address             x000063B4000400AA 
Error Count                       1. Error Occurrences This Entry 
 
----- Software Info ----- 
UCB$x_ERRCNT                      6. Errors This Unit

2.6. Synchronizing Cluster Functions by the Distributed Lock Manager

The distributed lock manager is an OpenVMS feature for synchronizing functions required by the distributed file system, the distributed job controller, device allocation, user-written OpenVMS Cluster applications, and other OpenVMS products and software components.

The distributed lock manager uses the connection manager and SCS to communicate information between OpenVMS Cluster computers.

2.6.1. Distributed Lock Manager Functions

The functions of the distributed lock manager include the following:
  • Synchronizes access to shared clusterwide resources, including:
    • Devices

    • Files

    • Records in files

    • Any user-defined resources, such as databases and memory

    Each resource is managed clusterwide by an OpenVMS Cluster computer.

  • Implements the $ENQ and $DEQ system services to provide clusterwide synchronization of access to resources by allowing the locking and unlocking of resource names.

    Reference: For detailed information about system services, refer to the VSI OpenVMS System Services Reference Manual.

  • Queues process requests for access to a locked resource. This queuing mechanism allows processes to be put into a wait state until a particular resource is available. As a result, cooperating processes can synchronize their access to shared objects, such as files and records.

  • Releases all locks that an OpenVMS Cluster computer holds if the computer fails. This mechanism allows processing to continue on the remaining computers.

  • Supports clusterwide deadlock detection.

2.6.2. System Management of the Lock Manager

The lock manager is fully automated and usually requires no explicit system management. However, the LOCKDIRWT and LOCKRMWT system parameters can be used to adjust the distribution of activity and control of lock resource trees across the cluster.

A lock resource tree is an abstract entity on which locks can be placed. Multiple lock resource trees can exist within a cluster. For every resource tree, there is one node known as the directory node and another node known as the lock resource master node.

A lock resource master node controls a lock resource tree and is aware of all the locks on the lock resource tree. All locking operations on the lock tree must be sent to the resource master. These locks can come from any node in the cluster. All other nodes in the cluster only know about their specific locks on the tree.

Furthermore, all nodes in the cluster have many locks on many different lock resource trees, which can be mastered on different nodes. When creating a new lock resource tree, the directory node must first be queried if a resource master already exists.

The LOCKDIRWT parameter allocates a node as the directory node for a lock resource tree. The higher a node's LOCKDIRWT setting, the higher the probability that it will be the directory node for a given lock resource tree.

For most configurations, large computers and boot nodes perform optimally when LOCKDIRWT is set to 1 and satellite nodes have LOCKDIRWT set to 0. These values are set automatically by the CLUSTER_CONFIG.COM procedure. Nodes with a LOCKDIRWT of 0 will not be the directory node for any resources unless all nodes in the cluster have a LOCKDIRWT of 0.

In some circumstances, you may want to change the values of the LOCKDIRWT parameter across the cluster to control the extent to which nodes participate as directory nodes.

LOCKRMWT influences which node is chosen to remaster a lock resource tree. Because there is a performance advantage for nodes mastering a lock resource tree (as no communication is required when performing a locking operation),the lock resource manager supports remastering lock trees to other nodes in the cluster. Remastering a lock resource tree means to designate another node in the cluster as the lock resource master for that lock resource tree and to move the lock resource tree to it.

A node is eligible to be a lock resource master node if it has locks on that lock resource tree. The selection of the new lock resource master node from the eligible nodes is based on each node's LOCKRMWT system parameter setting and each node's locking activity.

LOCKRMWT can contain a value between 0 and 10; the default is 5. The following list describes how the value of the LOCKRMWT system parameter affects resource tree mastery and how lock activity can affect the decision:
  • Any node that has a LOCKRMWT value of 0 will attempt to remaster a lock tree to another node which has locks on that tree, as long as the other node has a LOCKRMWT greater than 0.

  • Nodes with a LOCKRMWT value of 10 will be given resource trees from other nodes that have a LOCKRMWT less than 10.

  • Otherwise, the difference in LOCKRMWT is computed between the master and the eligible node. The higher the difference, the more activity is required by the eligible node for the lock tree to move.

In most cases, maintaining the default value of 5 for LOCKRMWT is appropriate, but there may be cases where assigning some nodes a higher or lower LOCKRMWT is useful for determining which nodes master a lock tree. The LOCKRMWT parameter is dynamic, hence it can be adjusted, if necessary.

2.6.3. Large-Scale Locking Applications

The Enqueue process limit (ENQLM), which is set in the SYSUAF.DAT file and which controls the number of locks that a process can own, can be adjusted to meet the demands of large scale databases and other server applications.

Prior to OpenVMS Version 7.1, the limit was 32767. This limit was removed to enable the efficient operation of large scale databases and other server applications. A process can now own up to 16,776,959 locks, the architectural maximum. By setting ENQLM in SYSUAF.DAT to 32767 (using the Authorize utility), the lock limit is automatically extended to the maximum of 16,776,959 locks. $CREPRC can pass large quotas to the target process if it is initialized from a process with the SYSUAF Enqlm quota of 32767.

Reference: See the VSI OpenVMS Programming Concepts Manual for additional information about the distributed lock manager and resource trees. See the VSI OpenVMS System Manager's Manual for more information about Enqueue Quota.

2.7. Resource Sharing

Resource sharing in an OpenVMS Cluster system is enabled by the distributed file system, RMS, and the distributed lock manager.

2.7.1. Distributed File System

The OpenVMS Cluster distributed file system allows all computers to share mass storage and files. The distributed file system provides the same access to disks, tapes, and files across the OpenVMS Cluster that is provided on a standalone computer.

2.7.2. RMS and the Distributed Lock Manager

The distributed file system and OpenVMS Record Management Services (RMS)use the distributed lock manager to coordinate clusterwide file access. RMS files can be shared to the record level.

Almost any disk or tape device can be made available to the entire OpenVMS Cluster system. The devices can be:
  • Connected to a supported storage subsystem

  • A local device that is served to the OpenVMS Cluster

All cluster-accessible devices appear as if they are connected to every computer.

2.8. Disk Availability

Locally connected disks can be served across an OpenVMS Cluster by the MSCP server.

2.8.1. MSCP Server

The MSCP server makes locally connected disks, including the following, available across the cluster:
  • DSA disks local to OpenVMS Cluster members using SDI

  • HSG and HSV disks in an OpenVMS Cluster using mixed interconnects

  • SCSI and HSZ disks

  • SAS, LSI 1068 SAS and LSI Logic 1068e SAS disks

  • FC and HSG disks

  • Disks on boot servers and disk servers located anywhere in the OpenVMS Cluster

In conjunction with the disk class driver (DUDRIVER), the MSCP server implements the storage server portion of the MSCP protocol on a computer, allowing the computer to function as a storage controller. The MSCP protocol defines conventions for the format and timing of messages sent and received for certain families of mass storage controllers and devices designed by VSI. The MSCP server decodes and services MSCP I/O requests sent by remote cluster nodes.

Note: The MSCP server is not used by a computer to access files on locally connected disks.

2.8.2. Device Serving

Once a device is set up to be served:
  • Any cluster member can submit I/O requests to it.

  • The local computer can decode and service MSCP I/O requests sent by remote OpenVMS Cluster computers.

2.8.3. Enabling the MSCP Server

The MSCP server is controlled by the MSCP_LOAD and MSCP_SERVE_ALL system parameters. The values of these parameters are set initially by answers to questions asked during the OpenVMS installation procedure (described in Section 8.4, “Changing Computer Characteristics”), or during the CLUSTER_CONFIG.COM procedure (described in Chapter 8, Configuring an OpenVMS Cluster System).

The default values for these parameters are as follows:
  • MSCP is not loaded on satellites.

  • MSCP is loaded on boot server and disk server nodes.

Reference: See Section 6.3, “MSCP and TMSCP Served Disks and Tapes” for more information about setting system parameters for MSCP serving.

2.9. Tape Availability

Locally connected tapes can be served across an OpenVMS Cluster by the TMSCP server.

2.9.1. TMSCP Server

The TMSCP server makes locally connected tapes, available across the cluster including the following:
  • HSG and HSV tapes

  • SCSI tapes

  • SAS tapes

The TMSCP server implements the TMSCP protocol, which is used to communicate with a controller for TMSCP tapes. In conjunction with the tape class driver (TUDRIVER), the TMSCP protocol is implemented on a processor, allowing the processor to function as a storage controller.

The processor submits I/O requests to locally accessed tapes, and accepts the I/O requests from any node in the cluster. In this way, the TMSCP server makes locally connected tapes available to all nodes in the cluster. The TMSCP server can also make HSG and HSV tapes accessible to OpenVMS Cluster satellites.

2.9.2. Enabling the TMSCP Server

The TMSCP server is controlled by the TMSCP_LOAD system parameter. The value of this parameter is set initially by answers to questions asked during the OpenVMS installation procedure (described in Section 4.2.3, “Information Required”) or during the CLUSTER_CONFIG.COM procedure(described in Section 8.4, “Changing Computer Characteristics”). By default, the setting of the TMSCP_LOAD parameter does not load the TMSCP server and does not serve any tapes.

2.10. Queue Availability

The distributed queue manager makes queues available across the cluster to achieve the following:

Function

Description

Permit users on any OpenVMS Cluster computer to submit batch and print jobs to queues that execute on any computer in the OpenVMS Cluster

Users can submit jobs to any queue in the cluster, provided that the necessary mass storage volumes and peripheral devices are accessible to the computer on which the job executes.

Distribute the batch and print processing work load over OpenVMS Cluster nodes

System managers can set up generic batch and print queues that distribute processing work loads among computers. The distributed queue manager directs batch and print jobs either to the execution queue with the lowest ratio of jobs-to-queue limit or to the next available printer.

The distributed queue manager uses the distributed lock manager to signal other computers in the OpenVMS Cluster to examine the batch and print queue jobs to be processed.

2.10.1. Controlling Queues

To control queues, you use one or several queue managers to maintain a clusterwide queue database that stores information about queues and jobs.

Reference: For detailed information about setting up OpenVMS Cluster queues, see Chapter 7, Setting Up and Managing Cluster Queues.

Chapter 3. OpenVMS Cluster Interconnect Configurations

This chapter provides an overview of various types of OpenVMS Cluster configurations and the ways they are interconnected.

For definitive information about supported OpenVMS Cluster configurations, see:
  • VSI OpenVMS Cluster Software Product Description

  • Guidelines for OpenVMS Cluster Configurations

3.1. Overview

Every node in an OpenVMS Cluster must have direct connections to all other nodes. Sites can choose to use one or more of the following interconnects:
  • LAN (Ethernet)

  • Internet Protocol (IP), Ethernet

  • MEMORY CHANNEL (Alpha only)

  • SMCI (Shared memory CI) (Alpha only) in OpenVMS Galaxy configurations, as described in the VSI OpenVMS Alpha Partitioning and Galaxy Guide

  • SCSI (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications for limited configurations)

  • Fibre Channel (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications)

  • SAS (supported only as a node-to-storage interconnect, requires a second interconnect for node-to-node (SCS) communications for limited configurations) (Integrity servers only)

Processing needs and available hardware resources determine how individual OpenVMS Cluster systems are configured. The configuration discussions in this chapter are based on these physical interconnects.

You can use bridges or switches to connect the OpenVMS server nodes to any intersite interconnect the WAN supplier provides, such as [D]WDM, Gigabit Ethernet, Fibre Channel or others.

Note

Multihost shared storage on a SCSI interconnect, commonly known as SCSI clusters, is not supported. It is also not supported on OpenVMS Alpha systems for newer SCSI adapters. However, multihost shared storage on industry-standard Fibre Channel is supported.

Locally attached storage, on both OpenVMS Alpha systems (FC or SCSI storage) and OpenVMS Integrity server systems (Fibre Channel, SAS, or SCSI storage), can be served to any other member of the cluster.

3.2. OpenVMS Cluster Systems Interconnected by LANs

All Ethernet interconnects are industry-standard local area networks that are generally shared by a wide variety of network consumers. When OpenVMS Cluster systems are based on LAN, cluster communications are carried out by a port driver (PEDRIVER) that emulates CI port functions. When configured for Clusters Over IP (COIP), also known as IP Cluster Interconnect (IPCI), the port driver (PEDRIVER) uses TCP/IP devices as cluster transport devices. For more details see Section

3.2.1. Design

The OpenVMS Cluster software is designed to use the LAN interconnects simultaneously with the DECnet, TCP/IP, and SCS protocols. This is accomplished by allowing LAN data link software to control the hardware port. This software provides a multiplexing function so that the cluster protocols are simply another user of a shared hardware resource. See Figure 2.1, “OpenVMS Cluster System Architecture” for an illustration of this concept.

3.2.1.1. PEDRIVER Fast Path Support

PEdriver, the software that enables OpenVMS Cluster communications over a LAN, also provides Fast Path support. This PEdriver feature provides the following benefits:
  • Improves SMP performance scalability.

  • Reduces the contention for the SCS/IOLOCK8 spinlock. PEdriver uses a private port mainline spinlock to synchronize its internal operation.

  • Allows PEdriver to perform cluster communications processing on a secondary CPU, thus offloading the primary CPU.

  • Allows PEdriver to process cluster communications using a single CPU.

  • Reduces CPU cost by providing a Fast Path streamlined code path for DSA and served blocked data operations.

For more detailed information, see the VSI OpenVMS I/O User's Reference Manual, the VSI OpenVMS System Manager's Manual, and the VSI OpenVMS System Management Utilities Reference Manual.

3.2.2. Cluster Group Numbers and Cluster Passwords

A single LAN can support multiple LAN-based OpenVMS Cluster systems. Each OpenVMS Cluster is identified and secured by a unique cluster group number and a cluster password. Chapter 2, OpenVMS Cluster Concepts describes cluster group numbers and cluster passwords in detail.

3.2.3. Servers

OpenVMS Cluster computers interconnected by a LAN are generally configured as either servers or satellites. The following table describes servers.
Server TypeDescription

MOP servers

Downline load the OpenVMS boot driver to satellites by means of the Maintenance Operations Protocol (MOP).

Disk servers

Use MSCP server software to make their locally connected disks available to satellites over the LAN.

Tape servers

Use TMSCP server software to make their locally connected tapes available to satellite nodes over the LAN.

Boot servers

A combination of a MOP server and a disk server that serves one or more Alpha system disks. Boot and disk servers make user and application data disks available across the cluster. These servers must be the most powerful computers in the OpenVMS Cluster and must use the highest-bandwidth LAN adapters in the cluster. Boot servers must always run the MSCP server software.

3.2.4. Satellites

Satellites are computers without a local system disk. Generally, satellites are consumers of cluster resources, although they can also provide facilities for disk serving, tape serving, and batch processing. If satellites are equipped with local disks, they can enhance performance by using such local disks for paging and swapping.

Satellites are booted remotely from a boot server (or from a MOP server and a disk server) serving the system disk. Section 3.2.5, “Satellite Booting (Alpha)” describes MOP and disk server functions during satellite booting.

3.2.5. Satellite Booting (Alpha)

When a satellite requests an operating system load, a MOP server for the appropriate OpenVMS Alpha operating system sends a bootstrap image to the satellite that allows the satellite to load the rest of the operating system from a disk server and join the cluster. The sequence of actions during booting is described in Table 3.1, “Satellite Booting Process”.
Table 3.1. Satellite Booting Process

Step

Action

Comments

1

Satellite requests MOP service.

This is the original boot request that a satellite sends out across the network. Any node in the OpenVMS Cluster that has MOP service enabled and has the LAN address of the particular satellite node in its database can become the MOP server for the satellite.

2

MOP server loads the Alpha system.

The MOP server responds to an Alpha satellite boot request by downline loading the SYS$SYSTEM:APB.EXE program along with the required parameters.

For Alpha computers, some of these parameters include:
  • System disk name

  • Root number of the satellite

3

Satellite finds additional parameters located on the system disk and root.

OpenVMS Cluster system parameters, such as SCSSYSTEMID, SCSNODE, an NISCS_CONV_BOOT. The satellite also finds the cluster group code and password.

4

Satellite executes the load program

The program establishes an SCS connection to a disk server for the satellite system disk and loads the SYSBOOT.EXE program.

Configuring and starting a satellite booting service for Alpha computers is described in detail in Section 4.5, “Configuring and Starting a Satellite Booting Service”.

3.2.6. Satellite Booting (Integrity servers)

Configuring and starting a satellite booting service for Integrity server systems is described in detail in Section 4.5, “Configuring and Starting a Satellite Booting Service”.

3.2.7. Configuring Multiple LAN Adapters

LAN support for multiple adapters allows PEDRIVER (the port driver for the LAN) to establish more than one channel between the local and remote cluster nodes. A channel is a network path between two nodes that is represented by a pair of LAN adapters.

3.2.7.1. System Characteristics

OpenVMS Cluster systems with multiple LAN adapters have the following characteristics:
  • At boot time, all Ethernet adapters are automatically configured for local area OpenVMS Cluster use. If Cluster over IP is configured, TCP/IP is started on IP-enabled LAN adapters and configured for cluster use.

  • PEDRIVER automatically detects and creates a new channel between the local node and each remote cluster node for each unique pair of LAN adapters.

  • Channel viability is monitored continuously.

  • In many cases, channel failure does not interfere with node-to-node (virtual circuit)communications as long as there is at least one remaining functioning channel between the nodes.

3.2.7.2. System Requirements

Configurations for OpenVMS Cluster systems with multiple LAN adapters must meet the following requirements:
  • The MOP server and the system disk server for a given satellite must be connected to the same extended LAN segment. (LANs can be extended using bridges that manage traffic between two or more local LANs).

  • All nodes must have a direct path to all other nodes. A direct path can be a bridged or a nonbridged LAN segment. When considering Cluster over IP configurations, a direct path can be via TCP/IP.

Rule: For each node, DECnet for OpenVMS (Phase IV) and MOP serving (Alpha or VAX, as appropriate) can be performed by only one adapter per extended LAN to prevent LAN address duplication.

3.2.7.3. Guidelines

The following guidelines are for configuring OpenVMS Cluster systems with multiple LAN adapters. If you configure these systems according to the guidelines, server nodes (nodes serving disks, tape, and lock traffic) can typically use some of the additional bandwidth provided by the added LAN adapters and increase the overall performance of the cluster. However, the performance increase depends on the configuration of your cluster and the applications it supports.

Configurations with multiple LAN adapters should follow these guidelines:
  • Connect each LAN adapter to a separate LAN segment. A LAN segment can be bridged or nonbridged. Doing this can help provide higher performance and availability in the cluster. The LAN segments can be Ethernet segments.

  • Distribute satellites equally among the LAN segments. Doing this can help to distribute the cluster load more equally across all of the LAN segments.

  • Systems providing MOP service should be distributed among the LAN segments to ensure that LAN failures do not prevent satellite booting. Systems should be bridged to multiple LAN segments for performance and availability.

  • For the number of LAN adapters supported per node, refer to the OpenVMS Cluster Software SPD.

3.2.8. LAN Examples

Figure 3.1, “LAN OpenVMS Cluster System with Single Server Node and System Disk” shows an OpenVMS Cluster system based on a LAN interconnect with a single OpenVMS server node and a single OpenVMS system disk.

Figure 3.1. LAN OpenVMS Cluster System with Single Server Node and System Disk
LAN OpenVMS Cluster System with Single Server Node and System Disk

In Figure 3.1, “LAN OpenVMS Cluster System with Single Server Node and System Disk”, the server node (and its system disk) is a single point of failure. If the server node fails, the satellite nodes cannot access any of the shared disks including the system disk. Note that some of the satellite nodes have locally connected disks. If you convert one or more of these into system disks, satellite nodes can boot from their own local system disk.

3.2.9. Fast Path for LAN Devices

With OpenVMS Version 7.3-2, further enhancements have been made to Fast Path for LAN devices, which will continue to help streamline I/O processing and improve symmetric-multiprocessing (SMP) performance scalability on newer Alpha Server systems. Enhancements include:
  • Reduced contention for the SCS/IOLOCK8 spinlock. The LAN drivers now synchronize using a LAN port-specific spinlock where possible.

  • Offload of the primary CPU. The LAN drivers may be assigned to a secondary CPU so that I/O processing can be initiated and completed on the secondary CPU. This offloads the primary CPU and reduces cache contention between processors.

These features enhance the Fast Path functionality that already exist in LAN drivers. The enhanced functionality includes additional optimizations, preallocating of resources, and providing an optimized code path for mainline code.

For more information, see the VSI OpenVMS I/O User's Reference Manual

3.2.10. LAN Bridge Failover Process

The following table describes how the bridge parameter settings can affect the failover process.

Option

Comments

Decreasing the LISTEN_TIME value allows the bridge to detect topology changes more quickly.

If you reduce the LISTEN_TIME parameter value, you should also decrease the value for the HELLO_INTERVAL bridge parameter according to the bridge-specific guidelines. However, note that decreasing the value for the HELLO_INTERVAL parameter causes an increase in network traffic.

Decreasing the FORWARDING_DELAY value can cause the bridge to forward packets unnecessarily to the other LAN segment.

Unnecessary forwarding can temporarily cause more traffic on both LAN segments until the bridge software determines which LAN address is on each side of the bridge.

Note

If you change a parameter on one LAN bridge, you should change that parameter on all bridges to ensure that selection of a new root bridge does not change he value of the parameter. The actual parameter value the bridge uses is the value specified by the root bridge.

3.2.11. Virtual LAN Support in OpenVMS

Virtual LAN (VLAN) is a mechanism for segmenting a LAN broadcast domain into smaller sections. The IEEE 802.1Q specification defines the operation and behavior of a VLAN. The OpenVMS implementation adds IEEE 802.1Q support to selected OpenVMS LAN drivers so that OpenVMS can now route VLAN tagged packets to LAN applications using a single LAN adapter.

You can use VLAN to do the following:
  • Segment specific LAN traffic on a network for the purposes of network security or traffic containment, or both.

  • Use VLAN isolated networks to simplify address management.

3.2.11.1. VLAN Design

In OpenVMS, VLAN presents a virtual LAN device to LAN applications. The virtual LAN device associates a single IEE 802.1Q tag with communications over a physical LAN device. The virtual device provides the ability to run any LAN application (for example, SCA, DECnet, TCP/IP, or LAT) over a physical LAN device, allowing host-to-host communications as shown in Figure 3.2, “Virtual LAN”.

Note

DECnet-Plus and DECnet Phase IV can be configured to run over a VLAN device.

Figure 3.2. Virtual LAN
Virtual LAN

OpenVMS VLAN has been implemented through a new driver, SYS$VLANDRIVER.EXE, which provides the virtual LAN devices. Also, existing LAN drivers have been updated to handle VLAN tags. LANCP.EXE and LANACP.EXE have been updated with the ability to create and deactivate VLAN devices and to display status and configuration information.

The OpenVMS VLAN subsystem was designed with particular attention to performance. Thus, the performance cost of using VLAN support is negligible.

When configuring VLAN devices, remember that VLAN devices share the same locking mechanism as the physical LAN device. For example, running OpenVMS cluster protocol on a VLAN device along with the underlying physical LAN device does not result in increased benefit and might, in fact, hinder performance.

3.2.11.2. VLAN Support Details

All supported Gigabit and 10-Gb (Integrity servers-only) LAN devices are capable of handling VLAN traffic on Alpha and Integrity server systems.

The following list describes additional details of VLAN-related support:
  • Switch support

    For VLAN configuration, the only requirement of a switch is conformance to the IEEE 802.1Q specification. The VLAN user interface to the switch is not standard; therefore, you must pay special attention when you configure as witch and especially when you configure VLANs across different switches.

  • LAN Failover support Figure 3.3, “LAN Failover Support” illustrates LAN Failover support.

    Figure 3.3. LAN Failover Support
    LAN Failover Support

    You can create VLAN devices using a LAN Failover set as a source if all members of the set are VLAN-capable devices. However, you cannot build a Failover set using VLAN devices.

  • Supported capabilities

    VLAN devices inherit the capability of the underlying physical LAN device, including fast path, auto-negotiation, and jumbo frame setting. If a capability needs to be modified, you must modify the underlying physical LAN device.

  • Restrictions

    No support exists for satellite booting over a VLAN device. The OpenVMS LAN boot drivers do not include VLAN support; therefore, you cannot use a VLAN device to boot an OpenVMS system. Currently, no support exists in OpenVMS for automatic configuration of VLAN devices. You must create VLAN devices explicitly using LANCP commands.

3.3. Cluster over IP

OpenVMS Version 8.4 has been enhanced with the Cluster over IP (Internet Protocol) feature. Cluster over IP provides the ability to form clusters beyond a single LAN or VLAN segment using industry standard Internet Protocol.

System managers also have the ability to manage or monitor OpenVMS cluster that uses IP for cluster communication using SCACP management utility.

Cluster protocol (SCS also known as SCA) over LAN is provided by Port Emulator driver (PEDRIVER). PEDRIVER uses User Datagram Protocol (UDP) and IP in addition to the native cluster LAN protocol for cluster communication as shown in Figure 3.4, “Cluster Communication Design Using IP”. The datagram characteristics of UDP combined with PEDRIVER's inbuilt reliable delivery mechanism is used for transporting cluster messages which is used by SYSAP (system level application) to communicate between two cluster nodes.

Cluster over IP is an optional feature that can be enabled in addition to the traditional LAN based communication. However, if both LAN and IP mode of communication exist between nodes in a cluster, PEDRIVER prefers LAN communication instead of IP.

Note

OpenVMS Cluster over IP and IP Cluster Interconnect (IPCI) terms are interchangeably used in the document and refers to using TCP/IP stack for cluster communication.

3.3.1. Design

Cluster over IP solution is an integration of the following:
  • PEDRIVER support for UDP protocol

  • TCP/IP Services boot time loading and initialization

Figure 3.4, “Cluster Communication Design Using IP” shows the cluster over IP architecture.

Figure 3.4. Cluster Communication Design Using IP
Cluster Communication Design Using IP

3.3.1.1. PEDRIVER Support for UDP

This consists of enhancing PEdriver to use the IP UDP protocol. Some of the features of this solution include:
  • The IP UDP service has the same packet delivery characteristics as 802 LANs. PEDRIVER implements the transport layer of NISCA which has inbuilt delay probing, reliable delivery for sequenced messages (retransmission), implement datagram service and also variable buffer size for block transfers for I/O suitable for cluster traffic.

  • The kernel VCI (KVCI) is a kernel mode. It acts as a highly efficient interface to the VSI OpenVMS TCP/IP Services stack. It is a variant of the VCI interface, which PEdriver uses to communicate with OpenVMS LAN drivers. PEDRIVER interfaces to UDP similar to a LAN device.

  • Only the lowest layer of PEDRIVER is extended to support UDP. The PEDRIVER changes are transparent to PEDRIVER's upper layers.

  • Providing management interface ability to control and configure IP interfaces to PEDRIVER.

3.3.1.2. TCP/IP Services Boot Time Loading and Initialization

To ensure that cluster communication is available in an IP only network environment, it is essential to have TCP/IP stack loaded when the cluster formation starts. This also retains the existing functionality of cluster formation of OpenVMS clusters. Normal booting sequence includes loading of LAN drivers followed by PEDRIVER. TCP/IP drivers are loaded when TCP/IP services are started. If cluster over IP is enabled, LAN, TCP/IP excelets, and PEDRIVER are loaded sequentially. Once the system comes up, TCP/IP services can be started to use other TCP/IP components, such as TELNET, FTP and so on.

Note

Ensure that the TCP/IP software is configured before configuring cluster over IP. To ensure that network and TCP/IP is configured properly, use the PING utility and ping the node from outside the subnet.

3.3.2. Availability

The ability to create a logical LAN failover set using IP for cluster communication provides high availability systems. The nodes will be able to resume if a local LAN card fails, as it will switchover to another interface configured in the logical LAN failover set. For a complete description of creating a logical LAN failover set, see Guidelines for OpenVMS Cluster Configurations. The hardware dependency on the LAN bridge is also overcome by GbE switches or routers used for transmission and forwarding the information.

3.3.3. System Characteristics

The existing functionalities of OpenVMS Clusters continue to exist with IP interconnect. Cluster over IP has the following characteristics:
  • Cluster over IP does not require any new hardware to use TCP/IP stack as interconnect.

  • UDP protocol is used for cluster communication.

  • The PEDRIVER includes delay probing technique that helps reduce latency in the IP network by selecting a path with the least latency.

  • The OpenVMS Cluster feature of rolling upgrades to the new version without a cluster reboot is retained.

  • Provides interoperability with servers running earlier versions of OpenVMS Clusters that are LAN based. Cluster over IP is available only with OpenVMS Version 8.4. Hence, if the node requires IP interconnect to be a part of the cluster, then all the nodes of the cluster must be running OpenVMS Version 8.4 and TCP/IP Services for OpenVMS, Version 5.7.

  • At the boot time, LAN, TCP/IP, and PEDRIVER are started sequentially.

  • PEDRIVER automatically detects and creates an IP channel for communication between two nodes.

  • Cluster over IP feature can be optionally enabled by running the CLUSTER_CONFIG_LAN.COM.

  • IP address used for cluster communication must be primary static address of the interface.

3.3.4. Software Requirements

The following software is required to support Clusters over IP interconnect:
  • OpenVMS Version 8.4 for Integrity servers or OpenVMS Alpha Version 8.4

  • TCP/IP Services for OpenVMS Version 5.7


Note

Ensure that the TCP/IP software is configured before configuring Cluster over IP. To ensure that network and TCP/IP is configured properly, use the PING utility and ping the node from outside the subnet.

3.3.5. Configuration Overview

IP Multicast Address

PEDRIVER uses 802 multicast for discovering cluster members in a LAN. IP multicast maps 1:1 onto the existing LAN discovery, and hence, has been selected as the preferred mechanism to discover nodes in a cluster. Every cluster using IP multicast will have one IP multicast address unique for that cluster. Multicast address is also used for keep-alive mechanism. Administratively scoped IP multicast address is used for cluster communication.

IP Unicast Address

Unicast address can be used if IP multicast is not enabled in a network. Remote node IP address must be present in the local node configuration files to allow the remote node to join the cluster. As a best practice, include all IP addresses and maintain one copy of the file throughout the cluster. $MC SCACP RELOAD can be used to refresh IP unicast list on a live system.

NISCS_USE_UDP SYSGEN Parameter

This parameter is set to enable the Cluster over IP functionality. PEDRIVER will use the UDP protocol in addition to IEEE 802.3 for cluster communication. CLUSTER_CONFIG_LAN is used to enable cluster over IP which will set this SYSGEN parameter.

UDP Port Number

UDP port number can be configured using CLUSTER_CONFIG_LAN and is constant in all nodes of a cluster.

Note

Standard internet practice such as firewall could be applied based on the port number that is selected for cluster.

3.3.5.1. Configuration Files

SYS$SYSTEM:PE$IP_CONFIG.DAT and SYS$SYSTEM:TCPIP$CLUSTER.DAT are the two configuration files. These files are loaded during the boot process and provide the necessary configuration details for Cluster over IP. Both these files are generated when a node is configured to be a member of the cluster and if cluster over IP is enabled during the configuration.

SYS$SYSTEM:PE$IP_CONFIG.DAT includes the optional IP multicast and IP unicast addresses of the nodes of the cluster. IP multicast messages are used for discovering a node within the same IP multicast domain. Remote nodes in a different IP multicast domain can use the IP unicast messaging technique to join the cluster. SYS$SYSTEM:PE$IP_CONFIG.DAT can be common for all the nodes of a cluster.

SYS$SYSTEM:TCPIP$CLUSTER.DAT contains the IP interface name and IP addresses on which cluster communication is enabled. It also includes the TCP/IP route information. SYS$SYSTEM:TCPIP$CLUSTER.DAT is unique for each node in a cluster.

3.3.6. Satellite Node Support

Integrity server satellite node support

The Integrity server satellite node must be in the same LAN on which the boot server resides. The Alpha satellite node must be in the same LAN as its disk server.

Alpha satellite node support

The Alpha console uses the MOP protocol for network load of satellite systems. Because the MOP protocol is non-routable, the satellite boot server or servers and all satellites booting from them must reside in the same LAN. In addition, the boot server must have at least one LAN device enabled for cluster communications to permit the Alpha satellite nodes to access the system disk.

3.3.7. High Availability Configuration using Logical LAN

The ability to create a logical LAN failover set and using IP for cluster communication with the logical LAN failover set provides high availability and can withstand NIC failure to provide high availability configuration. The nodes will be able to continue to communicate even if a local LAN card fails, as it will switchover to another interface configured in the logical LAN failover set. For a complete description of creating a logical LAN failover set and using it for Cluster over IP, see Guidelines for OpenVMS Cluster Configurations. For an example on how to create and configure a Logical LAN failover, refer to Scenario 5: Configuring an Integrity server Node Using a Logical LAN Failover set.

3.3.8. Performance Guidelines

For Cluster over IP configurations, the TCP/IP stack increases cluster communications overhead. Fast path CPU assignments for the LAN, TCP/IP, and PE devices can be adjusted if needed.

Note

Fast path configuration is not applicable for BG devices when (Packet Processing Engine) PPE is enabled. BG device always takes the primary CPU when cluster over IP is configured and if TCP/IP stack is loaded. It is required to move the BG device to an appropriate CPU using the $SET DEVICE/PREFERRED command.

3.3.9. Example

Figure 3.5, “OpenVMS Cluster Configuration Based on IP” illustrates an OpenVMS Cluster system based on IP as interconnect. Cluster over IP enables you to connect nodes that are located across various geographical locations. IP multicast is used to locate nodes in the same domain and IP unicast is used to locate nodes in different sites or domains. Cluster over IP supports mixed-architecture, that is, a combination of Integrity server systems and Alpha systems. Lab A and Lab B have the same IP multicast address, and are connected using different LANs.

Node A and Node B are located in the same LAN and use LAN for cluster communication. However, these nodes use IP for cluster communication with all other nodes that are geographically distributed in different sites.

Figure 3.5. OpenVMS Cluster Configuration Based on IP
OpenVMS Cluster Configuration Based on IP

3.4. OpenVMS Cluster Systems Interconnected by MEMORYCHANNEL (Alpha Only)

MEMORY CHANNEL is a high-performance cluster interconnect technology for PCI-based Alpha systems. With the benefits of very low latency, high bandwidth, and direct memory access, MEMORY CHANNEL complements and extends the ability of OpenVMS Clusters to work as a single virtual system. MEMORYCHANNEL is used for node-to-node cluster communications only. You use it in combination with another interconnect, such as Fibre Channel, SCSI, CI, or DSSI, that is dedicated to storage traffic.

3.4.1. Design

A node requires the following three hardware components to support a MEMORYCHANNEL connection:
  • PCI-to MEMORY CHANNEL adapter

  • Link cable (3 m or 10 feet long)

  • Port in a MEMORY CHANNEL hub (except for a two-node configuration in which the cable connects just two PCI adapters)

3.4.2. Examples

Figure 3.6, “Two-Node MEMORY CHANNEL OpenVMS Cluster Configuration” shows a two-node MEMORY CHANNEL cluster with shared access to Fibre Channel storage and a LAN interconnect for failover.

Figure 3.6. Two-Node MEMORY CHANNEL OpenVMS Cluster Configuration
Two-Node MEMORY CHANNEL OpenVMS Cluster Configuration

A three-node MEMORY CHANNEL cluster connected by a MEMORY CHANNEL hub and also by a LAN interconnect is shown in Figure 3.7, “Three-Node MEMORY CHANNEL OpenVMS Cluster Configuration”.The three nodes share access to the Fibre Channel storage. The LAN interconnect enables failover if the MEMORY CHANNEL interconnect fails.

Figure 3.7. Three-Node MEMORY CHANNEL OpenVMS Cluster Configuration
Three-Node MEMORY CHANNEL OpenVMS Cluster Configuration

3.5. Mixed-Interconnect OpenVMS Cluster Systems

A mixed-interconnect OpenVMS Cluster system is any OpenVMS Cluster system that uses more than one interconnect for SCS communication. You can use mixed interconnects to combine the advantages of each type and to expand your OpenVMS Cluster system. For example, an Ethernet cluster that requires more storage can expand with the addition of Fibre Channel, SCSI, or SAS connections.

Note

If any one node in a cluster requires IP for cluster communication, all the other members in the cluster must be enabled for IP cluster communication.

3.5.1. Availability

OpenVMS Cluster systems using a mix of interconnects provide maximum flexibility in combining CPUs, storage, and workstations into highly available configurations.

3.5.2. Examples

Figure 3.8, “OpenVMS Cluster System Using FC and Ethernet Interconnects” shows a mixed-interconnect OpenVMS Cluster system using both FC and Ethernet interconnects.

The computers based on the FC can serve HSG or HSV disks to the satellite nodes by means of MSCP server software and drivers; therefore, satellites can access the large amount of storage that is available through HSG and HSV subsystems.

Figure 3.8. OpenVMS Cluster System Using FC and Ethernet Interconnects
OpenVMS Cluster System Using FC and Ethernet Interconnects

3.6. Multihost SCSI OpenVMS Cluster Systems

OpenVMS Cluster systems support the SCSI as a storage interconnect. A SCSI interconnect, also called a SCSI bus, is an industry-standard interconnect that supports one or more computers, peripheral devices, and interconnecting components.

Beginning with OpenVMS Alpha Version 6.2, multiple Alpha computers using the KZPBA SCSI host-based adapter, can simultaneously access SCSI disks over a SCSI interconnect. Another interconnect, for example, a local area network, is required for host-to-host OpenVMS cluster communications. On Alpha computers, this support is limited to the KZPBA adapter. Newer SCSI host-based adapters for Alpha computers support only directly attached SCSI storage.

Beginning with OpenVMS Version 8.2-1, support is available for shared SCSI storage in a two-node OpenVMS Integrity server systems configuration using the MSA30-MI storage shelf.

Shared SCSI storage in an OpenVMS Cluster system enables computers connected to a single SCSI bus to share access to SCSI storage devices directly. This capability makes it possible to build highly available servers using shared access to SCSI storage.

3.6.1. Design for OpenVMS Alpha Configurations

Beginning with OpenVMS Alpha Version 6.2-1H3, OpenVMS Alpha supports up to three nodes on a shared SCSI bus as the storage interconnect. A quorum disk can be used on the SCSI bus to improve the availability of two-node configurations. Host-based RAID (including host-based shadowing) and the MSCP server are supported for shared SCSI storage devices.

Using the SCSI hub DWZZH-05, four nodes can be supported in a SCSI multihost OpenVMS Cluster system. In order to support four nodes, the hub's fair arbitration feature must be enabled.

For a complete description of these configurations, see Guidelines for OpenVMS Cluster Configurations.

3.6.2. Design for OpenVMS Integrity Server Shared SCSI Configurations

Shared SCSI storage in an OpenVMS Integrity server Cluster system is subject to the following restrictions:
  • Maximum of two OpenVMS Integrity server systems connected to a single SCSI bus.

  • Maximum of four shared-SCSI buses connected to each system.

  • rx1600 and rx2600 family systems are supported.

  • A7173A HBA is the only supported HBA.

  • MSA30-MI storage enclosure is the only supported SCSI storage type.

  • Ultra320 SCSI disk family is the only supported disk family.

In Figure 3.10, “Two-Node OpenVMS Integrity server Cluster System” the SCSI IDs of 6 and 7, are required in this configuration. One of the systems must have a SCSI ID of 6 for each A7173A adapter port connected to a shared SCSI bus, instead of the factory-set default of 7. You can use the U320_SCSI pscsi.efi utility, included in the IPF Offline Diagnostics and Utilities CD, to change the SCSIID. The procedure for doing this is documented in the HP A7173APCI-X Dual Channel Ultra 320 SCSI Host Bus Adapter Installation Guide.

3.6.3. Examples

Figure 3.9, “Three-Node OpenVMS Cluster Configuration Using a Shared SCSI Interconnect” shows an OpenVMS Cluster configuration that uses a SCSI interconnect for shared access to SCSI devices. Note that another interconnect, a LAN in this example, is used for host-to-host communications.

Figure 3.9. Three-Node OpenVMS Cluster Configuration Using a Shared SCSI Interconnect
Three-Node OpenVMS Cluster Configuration Using a Shared SCSI Interconnect

Figure 3.10, “Two-Node OpenVMS Integrity server Cluster System” illustrates the two-node OpenVMS Integrity server configuration. Note that a second interconnect, a LAN, is required for host-to-host OpenVMS Cluster communications. (OpenVMS Cluster communications are also known as SCA (System Communications Architecture) communications).

Figure 3.10. Two-Node OpenVMS Integrity server Cluster System
Two-Node OpenVMS Integrity server Cluster System

3.7. Serial Attached SCSI (SAS) (Integrity servers Only)

OpenVMS Cluster systems support SAS as a storage interconnect. SAS is a point-to-point architecture that transfers data to and from SCSI storage devices by using serial communication (one bit at a time). SAS uses the SAS devices and differential signaling method to achieve reliable, high-speed serial communication.

SAS combines high-end features from fiber channel (such as multi-initiator support and full duplex communication) and the physical interface leveraged from SATA (for better compatibility and investment protection), with the performance, reliability and ease of use of traditional SCSI technology.

3.8. Multihost Fibre Channel OpenVMS Cluster Systems

OpenVMS Cluster systems support FC interconnect as a storage interconnect. Fibre Channel is an ANSI standard network and storage interconnect that offers many advantages over other interconnects, including high-speed transmission and long interconnect distances. A second interconnect is required for node-to-node communications.

3.8.1. Design

OpenVMS Alpha supports the Fibre Channel SAN configurations described in the latest HP Storage Works SAN Design Reference Guide and in the Data Replication Manager (DRM) user documentation. This configuration support includes multiswitch Fibre Channel fabrics, up to 500 meters of multimode fiber, and up to 100 kilometers of single-mode fiber. In addition, DRM configurations provide long-distance intersite links (ISLs) through the use of the Open Systems Gateway and wave division multiplexors. OpenVMS supports sharing of the fabric and the HSG storage with non-OpenVMS systems.

OpenVMS provides support for the number of hosts, switches, and storage controllers specified in the Storage Works documentation. In general, the number of hosts and storage controllers is limited only by the number of available fabric connections.

Host-based RAID (including host-based shadowing) and the MSCP server are supported for shared Fibre Channel storage devices. Multipath support is available for these configurations.

For a complete description of these configurations, see Guidelines for OpenVMS Cluster Configurations.

Chapter 4. The OpenVMS Cluster Operating Environment

This chapter describes how to prepare the OpenVMS Cluster operating environment.

4.1. Preparing the Operating Environment

To prepare the cluster operating environment, there are a number of steps you perform on the first OpenVMS Cluster node before configuring other computers into the cluster. The following table describes these tasks.

Task

Section

Check all hardware connections to computer, interconnects, and devices.

Described in the appropriate hardware documentation.

Verify that all microcode and hardware is set to the correct revision levels.

Contact your support representative.

Install the OpenVMS operating system.

Section 4.2, “Installing the OpenVMS Operating System ”

Install all software licenses, including OpenVMS Cluster licenses.

Section 4.3, “Installing Software Licenses”

Install layered products.

Section 4.4, “Installing Layered Products”

Configure and start LANCP or DECnet for satellite booting

Section 4.5, “Configuring and Starting a Satellite Booting Service”

4.2. Installing the OpenVMS Operating System

Only one OpenVMS operating system version can exist on a system disk. Therefore, when installing or upgrading the OpenVMS operating systems ensure that you:
  • Install the OpenVMS Integrity servers operating system on each Integrity system disk

  • Install the OpenVMS Alpha operating system on each Alpha system disk

  • Install the OpenVMS x86-64 operating system on each x86-64 system disk


Note

Mixed architecture clusters of OpenVMS Integrity servers, Alpha systems, and x86-64 systems are supported.

4.2.1. System Disks

A system disk is one of the few resources that cannot be shared between Integrity and Alpha systems.

Once booted, Integrity server systems and Alpha systems can share access to data on any disk in the OpenVMS Cluster, including system disks. For example, an Integrity server system can mount an Alpha system disk as a data disk and an Alpha system can mount an Integrity server system disk as a data disk.

Note

An OpenVMS Cluster running both implementations of DECnet requires a system disk for DECnet for OpenVMS (Phase IV) and another system disk for DECnet-Plus (Phase V). For more information, see the DECnet-Plus documentation.

4.2.2. Where to Install

You might want to set up common system disks according to these guidelines:

IF you want the cluster to have...

THEN perform the installation or upgrade...

One common system disk for all computer members

Once on the cluster common system disk.

A combination of one or more common system disks and one or more local (individual) system disks

  • Once for each system disk

or
  • Once on a common system disk and then run the CLUSTER_CONFIG.COM procedure to create duplicate system disks (thus enabling systems to have their own local system disk)

Note: If your cluster includes multiple common system disks, you must later coordinate system files to define the cluster operating environment, as described in Chapter 5, Preparing a Shared Environment.

Reference: See Section 8.5, “Creating a Duplicate System Disk” for information about creating a duplicate system disk.

Example: If your OpenVMS Cluster consists of 10 computers, four of which boot from a common Integrity server system disk, two of which boot from a second common Integrity system disk, two of which boot from a common Alpha system disk, and two of which boot from their own local system disk, you need to perform an installation five times.

4.2.3. Information Required

Table 4.1, “Information Required to Perform an Installation” table lists the questions that the OpenVMS operating system installation procedure prompts you with and describes how certain system parameters are affected by responses you provide. You will notice that two of the prompts vary, depending on whether the node is running DECnet. The table also provides an example of an installation procedure that is taking place on a node named JUPITR.

Important: Be sure you determine answers to the questions before you begin the installation.

Note about versions: Refer to the appropriate OpenVMS Release Notes document for the required version numbers of hardware and firmware. When mixing versions of the operating system in an OpenVMS Cluster, check the release notes for information about compatibility.

Reference: Refer to the appropriate OpenVMS upgrade and installation manual for complete installation instructions.
Table 4.1. Information Required to Perform an Installation
PromptResponseParameter

Will this node be a cluster member (Y/N)?

WHEN you respond...

AND...

THEN the VAXcluster parameter is set to...

VAXCLUSTER

N

CI and DSSI hardware is not present

0 — Node will not participate in the OpenVMS Cluster.

N

CI and DSSI hardware is present

1 — Node will automatically participate in the OpenVMS Cluster in the presence of CI or DSSI hardware.

Y

 

2 — Node will participate in the OpenVMS Cluster.

What is the node's DECnet node name?

If the node is running DECnet, this prompt, the following prompt, and the SCSSYSTEMID prompt are displayed. Enter the DECnet node name or the DECnet–Plus node synonym (for example, JUPITR). If a node synonym is not defined, SCSNODE can be any name from 1 to 6 alphanumeric characters in length. The name cannot include dollar signs ($) or underscores (_).

SCSNODE

What is the node's DECnet node address?

Enter the DECnet node address (for example, a valid address might be 2.211). If an address has not been assigned, enter 0 now and enter a valid address when you start DECnet (discussed later in this chapter).

For DECnet–Plus, this question is asked when nodes are configured with a Phase IV compatible address. If a Phase IV compatible address is not configured, then the SCSSYSTEMID system parameter can be set to any value.

SCSSYSTEMID

What is the node's SCS node name?

If the node is not running DECnet, this prompt and the following prompt are displayed in place of the two previous prompts. Enter a name of 1 to 6 alphanumeric characters that uniquely names this node. At least 1 character must be a letter. The name cannot include dollar signs ($) or underscores (_).

SCSNODE

What is the node's SCSSYSTEMID number?

This number must be unique within this cluster. SCSSYSTEMID is the low-order 32 bits of the 48-bit system identification number.

If the node is running DECnet for OpenVMS, calculate the value from the DECnet address using the following formula:
  • SCSSYSTEMID = (DECnet-area-number * 1024)
  • + (DECnet-node-number)
Example: If the DECnet address is 2.211, calculate the value as follows:
SCSSYSTEMID = (2 * 1024) + 211 = 2259

SCSSYSTEMID

Will the Ethernet be used for cluster communications (Y/N)?

IF you respond...

THEN the NISCS_LOAD_PEA0 parameter is set to...

NISCS_LOAD_PEA0

N

0 — PEDRIVER is not loaded?; cluster communications does not use Ethernet or FDDI.

Y

1 — Loads PEDRIVER to enable cluster communications over Ethernet or FDDI.

Will the IP interconnect be used for cluster communications (Y/N)?

IF you respond...

THEN the NISCS_USE_UDP parameter is set to...

NISCS_USE_UDP

N

0 — Cluster over IP is disabled and uses the LAN interconnect for cluster communication

Y

1 — Cluster over IP is enabled and communicates using the TCP/IP stack. During the boot process, the TCP/IP driver and then the PEDRIVER authorization information is loaded for cluster communication. The hello packets are transmitted using IP multicast and unicast.

Enter this cluster's group number:

Enter a number in the range of 1 to 4095 or 61440 to 65535 (see Section 2.5, “OpenVMS Cluster Membership”). This value is stored in the CLUSTER_AUTHORIZE.DAT file in the SYS$COMMON:[SYSEXE] directory.

Not applicable

Enter this cluster's password:

Enter the cluster password. The password must be from 1 to 31 alphanumeric characters in length and can include dollar signs ($) and underscores (_) (see Section 2.5, “OpenVMS Cluster Membership”).This value is stored in scrambled form in the CLUSTER_AUTHORIZE.DAT file in the SYS$COMMON:[SYSEXE] directory.

Not applicable

Reenter this cluster's password for verification:

Reenter the password.

Not applicable

Will JUPITR be a disk server (Y/N)?

IF you respond...

THEN the MSCP_LOAD parameter is set to...

MSCP_LOAD

N

0 — The MSCP server will not be loaded. This is the correct setting for configurations in which all OpenVMS Cluster nodes can directly access all shared storage and do not require LAN failover.

Y

1 — Loads the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter, using the default CPU load capacity.

Will JUPITR serve HSC or RF disks (Y/N)?

IF you respond...

THEN the MSCP_SERVE_ALL parameter is set to...

MSCP_SERVE_ALL

Y

1 — Serves all available disks.

N

2 — Serves only locally connected (not HSC, HSJ, or RF) disks.

Enter a value for JUPITR's ALLOCLASS parameter:?

The value is dependent on the system configuration:
  • If the system disk is connected to a dual-pathed disk, enter a value from 1 to 255 that will be used on both storage controllers.

  • If the system is connected to a shared SCSI or SAS bus (it shares storage on that bus with another system) and if it does not use port allocation classes for naming the SCSI or SAS disks, enter a value from 1 to 255. This value must be used by all the systems and disks connected to the SCSI or SAS bus.

    Reference: For complete information about portal location classes, see Section 6.2.1, “Allocation Classes”.

  • If the system will use Volume Shadowing for OpenVMS, enter a value from 1 to 255.

    Reference: For more information, see the VSI OpenVMS Volume Shadowing Guide.

  • If none of the above are true, enter 0 (zero).

ALLOCLASS

Does this cluster contain a quorum disk [N]?

Enter Y or N, depending on your configuration. If you enter Y, the procedure prompts for the name of the quorum disk. Enter the device name of the quorum disk. (Quorum disks are discussed in Chapter 2, OpenVMS Cluster Concepts).

DISK_QUORUM

4.3. Installing Software Licenses

While rebooting at the end of the installation procedure, the system displays messages warning that you must install the operating system software and the OpenVMS Cluster software license. The OpenVMS Cluster software supports the OpenVMS License Management Facility (LMF). License units for clustered systems are allocated on an unlimited system-use basis.

4.3.1. Guidelines

Be sure to install all OpenVMS Cluster licenses and all licenses for layered products and DECnet as soon as the system is available. Procedures for installing licenses are described in the release notes distributed with the software kit and in the VSI OpenVMS License Management Utility Guide. Additional licensing information is described in the respective SPDs.

Use the following guidelines when you install software licenses:
  • Install an OpenVMS Cluster Software for Alpha license for each Alpha processor in the OpenVMS Cluster.

  • Install an OpenVMS Cluster Software for Integrity server system license for each Integrity server processor in the OpenVMS Cluster.

  • Install or upgrade licenses for layered products that runs on all nodes in an OpenVMS Cluster system.

  • OpenVMS Product Authorization Keys (PAKs) that have the Alpha option can be loaded and used only on Alpha processors. PAKs that have the Integrity servers option can be loaded and used only on Integrity server processors. However, PAKs can be located in a license database (LDB) that is shared by all processors(Integrity servers and Alpha).

  • PAK types, such as Activity PAKs (also known as concurrent or n-user PAKs)and Personal Use PAKs (identified by the RESERVE_UNITS option) work on Alpha systems.

  • PAK types, such as PCL PAKs (per core licensing) are only supported on Integrity servers.

  • License management commands can be issued from every node in the cluster.

4.4. Installing Layered Products

By installing layered products before other nodes are added to the OpenVMS Cluster, the software is installed automatically on new members when they are added to the OpenVMS Cluster system.

Note: For clusters with multiple system disks (Integrity servers or Alpha) you must perform a separate installation for each system disk.

4.4.1. Procedure

Table 4.2, “Installing Layered Products on a Common System Disk” describes the actions you take to install layered product on a common system disk.
Table 4.2. Installing Layered Products on a Common System Disk

Phase

Action

Before installation

Perform one or more of the following steps, as necessary for your system.
  1. Check each node's system parameters and modify the values, if necessary. Refer to the layered-product installation guide or release notes for information about adjusting system parameter values.

  2. If necessary, disable logins on each node that boots from the disk using the DCL command SET LOGINS/INTERACTIVE=0. Send a broadcast message to notify users about the installation.

Installation

Refer to the appropriate layered-product documentation for product-specific installation information. Perform the installation once for each system disk.

After installation

Perform one or more of the following steps, as necessary for your system.
  1. If necessary, create product-specific files in the SYS$SPECIFIC directory on each node. (The installation utility describes whether or not you need to create a directory in SYS$SPECIFIC.)When creating files and directories, be careful to specify exactly where you want the file to be located:
    • Use SYS$SPECIFIC or SYS$COMMON instead of SYS$SYSROOT.

    • Use SYS$SPECIFIC:[SYSEXE] or SYS$COMMON:[SYSEXE] instead of SYS$SYSTEM.

    Reference: Section 5.3, “Directory Structure on Common System Disks” describes directory structures in more detail.

  2. Modify files in SYS$SPECIFIC if the installation procedure tells you to do so. Modify files on each node that boots from this system disk.

  3. Reboot each node to ensure that:
    • The node is set up to run the layered product correctly.

    • The node is running the latest version of the layered product.

  4. Manually run the installation verification procedure (IVP) if you did not run it during the layered product installation. Run the IVP from at least one node in the OpenVMS Cluster, but preferably from all nodes that boot from this system disk.

4.5. Configuring and Starting a Satellite Booting Service

After you have installed the operating system and the required licenses on the first OpenVMS Cluster computer, you can configure and start a satellite booting service. You can use the LANCP utility, or DECnet software, or both.

VSI recommends LANCP for booting OpenVMS Cluster satellites. LANCP has shipped with the OpenVMS operating system since Version 6.2. It provides a general-purpose MOP booting service that can be used for booting satellites into an OpenVMS Cluster. (LANCP can service all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals, and can be used to customize your LAN environment).

DECnet provides a MOP booting service for booting OpenVMS Cluster satellites, as well as other local and wide area network services, including task-to-task communications for applications.

Note

If you plan to use LANCP in place of DECnet, and you also plan to move from DECnet PhaseIV to DECnet–Plus, VSI recommends the following order:
  1. Replace DECnet with LANCP for satellite booting (MOP downline load service) using LAN$POPULATE.COM.

  2. Migrate from DECnet Phase IV to DECnet-Plus.

There are two cluster configuration command procedures, CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. CLUSTER_CONFIG_LAN.COM uses LANCP to provide MOP services to boot satellites; CLUSTER_CONFIG.COM uses DECnet for the same purpose.

Before choosing LANCP, DECnet, or both, consider the following factors:
  • Applications you will be running on your cluster

    DECnet task-to-task communications is a method commonly used for communication between programs that run on different nodes in a cluster or a network. If you are running a program with that dependency, you need to run DECnet. If you are not running any programs with that dependency, you do not need to run DECnet.

  • Limiting applications that require DECnet to certain nodes in your cluster

    If you are running applications that require DECnet task-to-task communications, you can run those applications on a subset of the nodes in your cluster and restrict DECnet usage to those nodes. You can use LANCP software on the remaining nodes and use a different network, such as TCP/IP Services for OpenVMS, for other network services.

  • Managing two types of software for the same purpose

    If you are already using DECnet for booting satellites, you may not want to introduce another type of software for that purpose. Introducing any new software requires time to learn and manage it.

  • LANCP MOP services can coexist with DECnet MOP services in an OpenVMS Cluster in the following ways:
    • Running on different systems

      For example, DECnet MOP service is enabled on some of the systems on the LAN and LAN MOP is enabled on other systems.

    • Running on different LAN devices on the same system

      For example, DECnet MOP service is enabled on a subset of the available LAN devices on the system and LAN MOP is enabled on the remainder.

    • Running on the same LAN device on the same system but targeting a different set of nodes for service

      For example, both DECnet MOP and LAN MOP are enabled but LAN MOP has limited the nodes to which it will respond. This allows DECnet MOP to respond to the remaining nodes.

Instructions for configuring both LANCP and DECnet are provided in this section.

4.5.1. Configuring and Starting the LANCP Utility

You can use the LAN Control Program (LANCP) utility to configure a local area network (LAN). You can also use the LANCP utility, in place of DECnet or in addition to DECnet, to provide support for booting satellites in an OpenVMS Cluster and for servicing all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals.

Reference: For more information about using the LANCP utility to configure a LAN, see the VSI OpenVMS System Manager's Manual and the VSI OpenVMS System Management Utilities Reference Manual.

4.5.2. Booting Satellite Nodes with LANCP

The LANCP utility provides a general-purpose MOP booting service that can be used for booting satellites into an OpenVMS Cluster. It can also be used to service all types of MOP downline load requests, including those from terminal servers, LAN resident printers, and X terminals. To use LANCP for this purpose, all OpenVMS Cluster nodes must be running OpenVMS Version 6.2 or higher.

The CLUSTER_CONFIG_LAN.COM cluster configuration command procedure uses LANCP in place of DECnet to provide MOP services to boot satellites.

Note: If you plan to use LANCP in place of DECnet, and you also plan to move from DECnet for OpenVMS (Phase IV) to DECnet–Plus, VSI recommends the following order:
  1. Replace DECnet with LANCP for satellite booting (MOP downline load service), using LAN$POPULATE.COM.

  2. Migrate from DECnet for OpenVMS to DECnet–Plus.

4.5.3. Data Files Used by LANCP

LANCP uses the following data files:
  • SYS$SYSTEM:LAN$DEVICE_DATABASE.DAT

    This file maintains information about devices on the local node. By default, the file is created in SYS$SPECIFIC:[SYSEXE], and the system looks for the file in that location. However, you can modify the file name or location for this file by redefining the systemwide logical name LAN$DEVICE_DATABASE.

  • SYS$SYSTEM:LAN$NODE_DATABASE.DAT

    This file contains information about the nodes for which LANCP will supply boot service. This file must be shared among all nodes in the OpenVMS Cluster, including Integrity servers, Alpha, and VAX systems. By default, the file is created in SYS$COMMON:[SYSEXE], and the system looks for the file in that location. However, you can modify the file name or location for this file by redefining the systemwide logical name LAN$NODE_DATABASE.

4.5.4. Using LAN MOP Services in New Installations

To use LAN MOP services for satellite booting in new installations, follow these steps:
  1. Add the startup command for LANCP.

    You should start up LANCP as part of your system startup procedure. To do this, remove the comment from the line in SYS$MANAGER:SYSTARTUP_VMS.COM that runs the LAN$STARTUP command procedure. If your OpenVMS Cluster system will have more than one system disk, see Section 4.5.3, “Data Files Used by LANCP ” for a description of logicals that can be defined for locating LANCP configuration files.
    $ @SYS$STARTUP:LAN$STARTUP

    You should now either reboot the system or invoke the preceding command procedure from the system manager's account to start LANCP.

  2. Follow the steps in Chapter 8, Configuring an OpenVMS Cluster System for configuring an OpenVMS Cluster system and adding satellites. Use the CLUSTER_CONFIG_LAN.COM command procedure instead of CLUSTER_CONFIG.COM. If you invoke CLUSTER_CONFIG.COM, it gives you the option to switch to running CLUSTER_CONFIG_LAN.COM if the LANCP process has been started.

4.5.5. Using LAN MOP Services in Existing Installations

To migrate from DECnet MOP services to LAN MOP services for satellite booting, follow these steps:
  1. Redefine the LANCP database logical names.

    This step is optional. If you want to move the data files used by LANCP, LAN$DEVICE_DATABASE and LAN$NODE_DATABASE, off the system disk, redefine their systemwide logical names. Add the definitions to the system startup files.

  2. Use LANCP to create the LAN$DEVICE_DATABASE

    The permanent LAN$DEVICE_DATABASE is created when you issue the first LANCP DEVICE command. To create the database and get a list of available devices, enter the following commands:
    $ MCR LANCP
    LANCP> LIST DEVICE /MOPDLL
    %LANCP-I-FNFDEV, File not found, LAN$DEVICE_DATABASE
    %LANACP-I-CREATDEV, Created LAN$DEVICE_DATABASE file
    
    Device Listing, permanent database:
      --- MOP Downline Load Service Characteristics ---
    Device    State   Access Mode      Client            Data Size
    ------    -----   -----------      ------            ---------
    ESA0    Disabled NoExlusive  NoKnownClientsOnly     246 bytes
    FCA0    Disabled NoExlusive  NoKnownClientsOnly     246 bytes
  3. Use LANCP to enable LAN devices for MOP booting.

    By default, the LAN devices have MOP booting capability disabled. Determine the LAN devices for which you want to enable MOP booting. Then use the DEFINE command in the LANCP utility to enable these devices to service MOP boot requests in the permanent database, as shown in the following example:
    LANCP> DEFINE DEVICE ESA0:/MOP=ENABLE
  4. Run LAN$POPULATE.COM (found in SYS$EXAMPLES) to obtain MOP booting information and to produce LAN$DEFINE and LAN$DECNET_MOP_CLEANUP, which are site specific.

    LAN$POPULATE extracts all MOP booting information from a DECnet Phase IVNETNODE_REMOTE.DAT file or from the output of the DECnet–Plus NCL command SHOW MOP CLIENT * ALL.

    For DECnet Phase IV sites, the LAN$POPULATE procedure scans all DECnet areas (1–63) by default. If you MOP boot systems from only a single or a few DECnet areas, you can cause the LAN$POPULATE procedure to operate on a single area at a time by providing the area number as the P1 parameter to the procedure, as shown in the following example (including log):
    $ @SYS$EXAMPLES:LAN$POPULATE 15
     LAN$POPULATE - V1.0
    
     Do you want help (Y/N) <N>:
    
     LAN$DEFINE.COM has been successfully created.
    
     To apply the node definitions to the LANCP permanent database,
     invoke the created LAN$DEFINE.COM command procedure.
    
            VSI recommends that you review LAN$DEFINE.COM and remove any
            obsolete entries prior to executing this command procedure.
    
     A total of 2 MOP definitions were entered into LAN$DEFINE.COM
  5. Run LAN$DEFINE.COM to populate LAN$NODE_DATABASE.

    LAN$DEFINE populates the LANCP downline loading information into the LAN node database, SYS$COMMON:[SYSEVE]LAN$NODE_DATABASE.DAT file. VSI recommends that you review LAN$DEFINE.COM and remove any obsolete entries before executing it.

    In the following sequence, the LAN$DEFINE.COM procedure that was just created is displayed on the screen and then executed:
    $ TYPE LAN$DEFINE.COM
     $ !
     $ ! This file was generated by LAN$POPULATE.COM on 16-DEC-1996 09:20:31
     $ ! on node CLU21.
     $ !
     $ ! Only DECnet Area 15 was scanned.
     $ !
     $ MCR LANCP
     Define Node PORK    /Address=08-00-2B-39-82-85 /File=APB.EXE -
                      /Root=$21$DKA300:<SYS11.> /Boot_type=Alpha_Satellite
     Define Node JYPIG   /Address=08-00-2B-A2-1F-81 /File=APB.EXE -
                      /Root=$21$DKA300:<SYS10.> /Boot_type=Alpha_Satellite
     EXIT
    $ @LAN$DEFINE
     %LANCP-I-FNFNOD, File not found, LAN$NODE_DATABASE
     -LANCP-I-CREATNOD, Created LAN$NODE_DATABASE file
    $
    The following example shows a LAN$DEFINE.COM command procedure that was generated by LAN$POPULATE for migration from DECnet–Plus to LANCP.
    $ ! LAN$DEFINE.COM - LAN MOP Client Setup
    $ !
    $ ! This file was generated by LAN$POPULATE.COM at  8-DEC-1996 14:28:43.31
    $ ! on node BIGBOX.
    $ !
    $ SET NOON
    $ WRITE SYS$OUTPUT "Setting up MOP DLL clients in LANCP...
    $ MCR LANCP
    SET    NODE SLIDER
    /ADDRESS=08-00-2B-12-D8-72/ROOT=BIGBOX$DKB0:<SYS10.>/BOOT_TYP
    E=VAX_satellite/FILE=NISCS_LOAD.EXE
    DEFINE NODE SLIDER
    /ADDRESS=08-00-2B-12-D8-72/ROOT=BIGBOX$DKB0:<SYS10.>/BOOT_TYP
    E=VAX_satellite/FILE=NISCS_LOAD.EXE
    EXIT
    $ !
    $  WRITE SYS$OUTPUT "DECnet Phase V to LAN MOPDLL client migration complete!"
    $  EXIT 
  6. Run LAN$DECNET_MOP_CLEANUP.COM.

    You can use LAN$DECNET_MOP_CLEANUP.COM to remove the clients' MOP downline loading information from the DECnet database. VSI recommends that you review LAN$DECNET_MOP_CLEANUP.COM and remove any obsolete entries before executing it.

    The following example shows a LAN$DECNET_MOP_CLEANUP.COM command procedure that was generated by LAN$POPULATE for migration from DECnet–Plus to LANCP.

    Note: When migrating from DECnet–Plus, additional cleanup is necessary. You must edit your NCL scripts (*.NCL) manually.
    $ ! LAN$DECNET_MOP_CLEANUP.COM - DECnet MOP Client Cleanup
    $ !
    $ ! This file was generated by LAN$POPULATE.COM at  8-DEC-1995 14:28:43.47
    $ ! on node BIGBOX.
    $ !
    $ SET NOON
    $ WRITE SYS$OUTPUT "Removing MOP DLL clients from DECnet database..."
    $ MCR NCL
    DELETE NODE 0 MOP CLIENT SLIDER
    EXIT
    $ !
    $  WRITE SYS$OUTPUT "DECnet Phase V MOPDLL client cleanup complete!"
    $  EXIT 
  7. Start LANCP.

    To start LANCP, execute the startup command procedure as follows:
    $ @SYS$STARTUP:LAN$STARTUP
    %RUN-S-PROC_ID, identification of created process is 2920009B
    $
    You should start up LANCP for all boot nodes as part of your system startup procedure. To do this, include the following line in your site-specific startup file (SYS$MANAGER:SYSTARTUP_VMS.COM):
    $ @SYS$STARTUP:LAN$STARTUP

    If you have defined logicals for either LAN$DEVICE_DATABASE or LAN$NODE_DATABASE, be sure that these are defined in your startup files prior to starting up LANCP.

  8. Disable DECnet MOP booting.

    If you use LANCP for satellite booting, you may no longer need DECnet to handle MOP requests. If this is the case for your site, you can turn off this capability with the appropriate NCP command (DECnet for OpenVMS) or NCL commands (DECnet–Plus).

For more information about the LANCP utility, see the VSI OpenVMS System Manager's Manual and the VSI OpenVMS System Management Utilities Reference Manual.

4.5.6. Configuring DECnet

The process of configuring the DECnet network typically entails several operations, as shown in Table 4.3, “Procedure for Configuring the DECnet Network”.An OpenVMS Cluster running both implementations of DECnet requires a system disk for DECnet for OpenVMS (Phase IV) and another system disk for DECnet–Plus (Phase V).

Note: DECnet for OpenVMS implements Phase IV of Digital Network Architecture (DNA). DECnet–Plus implements Phase V of DNA. The following discussions are specific to the DECnet for OpenVMS product.

Reference: Refer to the DECnet–Plus documentation for equivalent DECnet–Plus configuration information.
Table 4.3. Procedure for Configuring the DECnet Network
StepAction

1

Log in as system manager and execute the NETCONFIG.COM command procedure as shown. Enter information about your node when prompted. Note that DECnet–Plus nodes execute the NET$CONFIGURE.COM command procedure.

Reference: See the DECnet for OpenVMS or the DECnet–Plus documentation, as appropriate, for examples of these procedures.

2

When a node uses multiple LAN adapter connections to the same LAN and also uses DECnet for communications, you must disable DECnet use of all but one of the LAN devices.

To do this, remove all but one of the lines and circuits associated with the adapters connected to the same LAN or extended LAN from the DECnet configuration database after the NETCONFIG.COM procedure is run.

For example, issue the following commands to invoke NCP and disable DECnet use of the LAN device XQB0:
$ RUN SYS$SYSTEM:NCP
NCP> PURGE CIRCUIT QNA-1 ALL
NCP> DEFINE CIRCUIT QNA-1 STA OFF
NCP> EXIT

References:

See Guidelines for OpenVMS Cluster Configurations for more information about distributing connections to LAN segments in OpenVMS Cluster configurations.

See the DECnet–Plus documentation for information about removing routing circuits associated with all but one LAN adapter. (Note that the LAN adapter issue is not a problem if the DECnet–Plus node uses extended addressing and does not have any Phase IV compatible addressing in use on any of the routing circuits).

3

Make remote node data available clusterwide. NETCONFIG.COM creates in the SYS$SPECIFIC:[SYSEXE] directory the permanent remote-node database file NETNODE_REMOTE.DAT, in which remote-node data is maintained. To make this data available throughout the OpenVMS Cluster, you move the file to the SYS$COMMON:[SYSEXE] directory.

Example: Enter the following commands to make DECnet information available clusterwide:
$ RENAME SYS$SPECIFIC:[SYSEXE]NETNODE_REMOTE.DAT
SYS$COMMON:[SYSEXE]NETNODE_REMOTE.DAT
If your configuration includes multiple system disks, you can set up a common NETNODE_REMOTE.DAT file automatically by using the following command in SYLOGICALS.COM:
$ DEFINE/SYSTEM/EXE NETNODE_REMOTE
ddcu:[directory]NETNODE_REMOTE.DAT

Notes: VSI recommends that you set up a common NETOBJECT.DAT file clusterwide in the same manner.

DECdns is used by DECnet–Plus nodes to manage node data (the namespace). For DECnet–Plus, Session Control Applications replace objects.

4

Designate and enable router nodes to support the use of a cluster alias. At least one node participating in a cluster alias must be configured as a level 1 router.

On Integrity servers and Alpha systems, you might need to enable level 1 routing manually because the NETCONFIG.COM procedure does not prompt you with the routing question.

Depending on whether the configuration includes a combination of Integrity sever nodes and Alpha nodes, you must enable level 1 routing manually (see the example below) on one of the Alpha nodes.

Example: On Alpha systems, if you need to enable level 1 routing on Alpha node, invoke the NCP utility to do so. For example:
$ RUN SYS$SYSTEM:NCP
NCP> DEFINE EXECUTOR TYPE ROUTING IV

Note: On Integrity servers and Alpha systems, level 1 routing is supported to enable cluster alias operations only.

5

Optionally, define a cluster alias. If you want to define a cluster alias, invoke the NCP utility to do so. The information you specify using these commands is entered in the DECnet permanent executor database and takes effect when you start the network.

Example: The following NCP commands establish SOLAR as an alias:
$ RUN SYS$SYSTEM:NCP
NCP> DEFINE NODE 2.1 NAME SOLAR
NCP> DEFINE EXECUTOR ALIAS NODE SOLAR
NCP> EXIT
$ 

Reference: Section 4.5.8, “What is Cluster Alias?” describes the cluster alias. Section 4.5.9, “Enabling Alias Operations” describes how to enable alias operations for other computers. See the DECnet–Plus documentation for information about setting up a cluster alias on DECnet–Plus nodes.

Note: DECnet for OpenVMS nodes and DECnet–Plus nodes cannot share a cluster alias.

4.5.7. Starting DECnet

If you are using DECnet–Plus, a separate step is not required to start the network. DECnet–Plus starts automatically on the next reboot after the node has been configured using the NET$CONFIGURE.COM procedure.

If you are using DECnet for OpenVMS, at the system prompt, enter the following command to start the network:
$ @SYS$MANAGER:STARTNET.COM

To ensure that the network is started each time an OpenVMS Cluster computer boots, add that command line to the appropriate startup command file or files. (Startup command files are discussed in Section 5.5, “Coordinating Startup Command Procedures”).

4.5.8. What is Cluster Alias?

The cluster alias acts as a single network node identifier for an OpenVMS Cluster system. When enabled, the cluster alias makes all the OpenVMS Cluster nodes appear to be one node from the point of view of the rest of the network.

Computers in the cluster can use the alias for communications with other computers in a DECnet network. For example, networked applications that use the services of an OpenVMS Cluster should use an alias name. Doing so ensures that the remote access will be successful when at least one OpenVMS Cluster member is available to process the client program's requests.

Rules:
  • DECnet for OpenVMS (Phase IV) allows a maximum of 64 OpenVMS Cluster computers to participate in a cluster alias. If your cluster includes more than 64 computers, you must determine which 64 should participate in the alias and then define the alias on those computers.

    At least one of the OpenVMS Cluster nodes that uses the alias node identifier must have level 1 routing enabled.
    • On Integrity servers and Alpha nodes, routing between multiple circuits is not supported. However, routing is supported to allow cluster alias operations. Level 1 routing is supported only for enabling the use of a cluster alias. The DVNETEXT PAK must be used to enable this limited function.

    • On Integrity servers, Alpha, and VAX systems, all cluster nodes sharing the same alias node address must be in the same area.

  • DECnet–Plus allows a maximum of 96 OpenVMS Cluster computers to participate in the cluster alias.

    DECnet–Plus does not require that a cluster member be a routing node, but an adjacent Phase V router is required to use a cluster alias for DECnet–Plus systems.

  • A single cluster alias can include nodes running either DECnet for OpenVMS or DECnet–Plus, but not both.

4.5.9. Enabling Alias Operations

If you have defined a cluster alias and have enabled routing as shown in Section 4.5.6, “Configuring DECnet”, you can enable alias operations for other computers after the computers are up and running in the cluster. To enable such operations (that is, to allow a computer to accept incoming connect requests directed toward the alias), follow these steps:
  1. Log in as system manager and invoke the SYSMAN utility. For example:
    $ RUN SYS$SYSTEM:SYSMAN
    SYSMAN> 
  2. At the SYSMAN> prompt, enter the following commands:
    SYSMAN> SET ENVIRONMENT/CLUSTER
    %SYSMAN-I-ENV, current command environment:
            Clusterwide on local cluster
            Username SYSTEM  will be used on nonlocal nodes
    SYSMAN> SET PROFILE/PRIVILEGES=(OPER,SYSPRV)
    SYSMAN> DO MCR NCP SET EXECUTOR STATE OFF
    %SYSMAN-I-OUTPUT, command execution on node X...
       .
       .
       .
    SYSMAN> DO MCR NCP DEFINE EXECUTOR ALIAS INCOMING ENABLED
    %SYSMAN-I-OUTPUT, command execution on node X...
       .
       .
       .
    SYSMAN> DO @SYS$MANAGER:STARTNET.COM
    %SYSMAN-I-OUTPUT, command execution on node X...
       .
       .
       .

Note: VSI does not recommend enabling alias operations for satellite nodes.

Reference: For more details about DECnet for OpenVMS networking and cluster alias, see the VSI OpenVMS DECnet Networking Manual and VSI OpenVMS DECnet Network Management Utilities. For equivalent information about DECnet–Plus, see the DECnet–Plus documentation.

4.5.10. Configuring TCP/IP

For information on how to configure and start TCP/IP, see the VSI TCP/IP Services for OpenVMS Installation and Configuration and the HP TCP/IP Services for OpenVMS Version 5.7 Release Notes.

Chapter 5. Preparing a Shared Environment

In any OpenVMS Cluster environment, it is best to share resources as much as possible. Resource sharing facilitates workload balancing because work can be distributed across the cluster.

5.1. Shareable Resources

Most, but not all, resources can be shared across nodes in an OpenVMS Cluster. The following table describes resources that can be shared.
Shareable ResourcesDescription

System disks

All members of the same architecture?can share a single system disk, each member can have its own system disk, or members can use a combination of both methods.

Data disks

All members can share any data disks. For local disks, access is limited to the local node unless you explicitly setup the disks to be cluster accessible by means of the MSCP server.

Tape drives

All members can share tape drives. (Note that this does not imply that all members can have simultaneous access.)For local tape drives, access is limited to the local node unless you explicitly set up the tapes to be cluster accessible by means of the TMSCP server. Only DSA tapes can be served to all OpenVMS Cluster members.

Batch and print queues

Users can submit batch jobs to any queue in the OpenVMS Cluster, regardless of the processor on which the job will actually execute. Generic queues can balance the load among the available processors.

Applications

Most applications work in an OpenVMS Cluster just as they do on a single system. Application designers can also create applications that run simultaneously on multiple OpenVMS Cluster nodes, which share data in a file.

User authorization files

All nodes can use either a common user authorization file (UAF)for the same access on all systems or multiple UAFs to enable node-specific quotas. If a common UAF is used, all user passwords, directories, limits, quotas, and privileges are the same on all systems.

5.1.1. Local Resources

The following table lists resources that are accessible only to the local node.
Nonshareable ResourcesDescription

Memory

Each OpenVMS Cluster member maintains its own memory.

User processes

When a user process is created on an OpenVMS Cluster member, the process must complete on that computer, using local memory.

Printers

A printer that does not accept input through queues is used only by the OpenVMS Cluster member to which it is attached. A printer that accepts input through queues is accessible by any OpenVMS Cluster member.

5.1.2. Sample Configuration

Figure 5.1, “Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)” shows an OpenVMS Cluster system that shares FC SAN storage between the Integrity servers and Alpha systems. Each architecture has its own system disk.

Figure 5.1. Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)
Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)

5.1.3. Storage in a Mixed-Architecture Cluster

This section describes the rules pertaining to storage, including system disks, in a mixed-architecture cluster consisting of OpenVMS Integrity servers and OpenVMS Alpha systems.

Figure 5.2, “Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)” is a simplified version of a mixed-architecture cluster of OpenVMS Integrity servers and OpenVMS Alpha systems with locally attached storage and a shared Storage Area Network (SAN).

Figure 5.2. Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)
Resource Sharing in Mixed-Architecture Cluster System (Integrity servers and Alpha)
Integrity server systems in a mixed-architecture OpenVMS Cluster system:
  • Must have an Integrity server system disk, either a local disk or a shared Fibre Channel disk.

  • Can use served Alpha disks and served Alpha tapes.

  • Can use SAN disks and tapes.

  • Can share the same SAN data disk with Alpha systems.

  • Can serve disks and tapes to other cluster members, both Integrity servers and Alpha systems.

Alpha systems in a mixed-architecture OpenVMS Cluster system:
  • Must have an Alpha system disk, which can be shared with other clustered Alpha systems.

  • Can use locally attached tapes and disks.

  • Can serve disks and tapes to both Integrity servers and Alpha systems.

  • Can use Integrity servers served data disks.

  • Can use SAN disks and tapes.

  • Can share the same SAN data disk with Integrity server systems.

5.2. Common-Environment and Multiple-Environment Clusters

Depending on your processing needs, you can prepare either an environment in which all environmental files are shared clusterwide or an environment in which some files are shared clusterwide while others are accessible only by certain computers.

The following table describes the characteristics of common- and multiple-environment clusters.
Cluster TypeCharacteristicsAdvantages

Common environment

Operating environment is identical on all nodes in the OpenVMS Cluster.

The environment is set up so that:
  • All nodes run the same programs, applications, and utilities.

  • All users have the same type of user accounts, and the same logical names are defined.

  • All users can have common access to storage devices and queues.(Note that access is subject to how access control list [ACL] protection is set up for each user.)

  • All users can log in to any node in the configuration and work in the same environment as all other users.

Easier to manage because you use a common version of each system file.

Multiple environment

Operating environment can vary from node to node.

An individual processor or a subset of processors are set up to:
  • Provide multiple access according to the type of tasks users perform and the resources they use.

  • Share a set of resources that are not available on other nodes.

  • Perform specialized functions using restricted resources while other processors perform general time sharing work.

  • Allow users to work in environments that are specific to the node where they are logged in.

Effective when you want to share some data among computers but you also want certain computers to serve specialized needs.

5.3. Directory Structure on Common System Disks

The installation or upgrade procedure for your operating system generates a common system disk, on which most operating system and optional product files are stored in a system root directory.

5.3.1. Directory Roots

The system disk directory structure is the same on Integrity servers and Alpha systems. Whether the system disk is for an Integrity server system or Alpha, the entire directory structure—that is, the common root plus each computer's local root is stored on the same disk. After the installation or upgrade completes, you use the CLUSTER_CONFIG.COM or CLUSTER_CONFIG_LAN.COM command procedure described in Chapter 8, Configuring an OpenVMS Cluster System to create a local root for each new computer to use when booting into the cluster.

In addition to the usual system directories, each local root contains a [SYS n.SYSCOMMON] directory that is a directory alias for [VMS$COMMON], the cluster common root directory in which cluster common files actually reside. When you add a computer to the cluster, the com procedure defines the common root directory alias.

5.3.2. Directory Structure Example

Figure 5.3, “Directory Structure on a Common System Disk” illustrates the directory structure set up for computers JUPITR and SATURN, which are run from a common system disk. The disk's master file directory (MFD) contains the local roots (SYS0 for JUPITR, SYS1 for SATURN) and the cluster common root directory, [VMS$COMMON].

Figure 5.3. Directory Structure on a Common System Disk
Directory Structure on a Common System Disk

5.3.3. Search Order

The logical name SYS$SYSROOT is defined as a search list that points first to a local root (SYS$SYSDEVICE:[SYS0.SYSEXE]) and then to the common root (SYS$COMMON:[SYSEXE]). Thus, the logical names for the system directories (SYS$SYSTEM, SYS$LIBRARY, SYS$MANAGER, and so forth) point to two directories.

Figure 5.4, “File Search Order on Common System Disk” shows how directories on a common system disk are searched when the logical name SYS$SYSTEM is used in file specifications.

Figure 5.4. File Search Order on Common System Disk
File Search Order on Common System Disk

Important: Keep this search order in mind when you manipulate system files on a common system disk. Computer-specific files must always reside and be updated in the appropriate computer's system subdirectory.

Examples
  1. MODPARAMS.DAT must reside in SYS$SPECIFIC:[SYSEXE], which is[SYS0.SYSEXE] on JUPITR, and in [SYS1.SYSEXE] on SATURN. Thus, to create a new MODPARAMS.DAT file for JUPITR when logged in on JUPITR, enter the following command:
    $ EDIT SYS$SPECIFIC:[SYSEXE]MODPARAMS.DAT
    Once the file is created, you can use the following command to modify it when logged on to JUPITR:
    $ EDIT SYS$SYSTEM:MODPARAMS.DAT

    Note that if a MODPARAMS.DAT file does not exist in JUPITR's SYS$SPECIFIC:[SYSEXE] directory when you enter this command, but there is a MODPARAMS.DAT file in the directory SYS$COMMON:[SYSEXE], the command edits the MODPARAMS.DAT file in the common directory. If there is no MODPARAMS.DAT file in either directory, the command creates the file in JUPITR's SYS$SPECIFIC:[SYSEXE] directory.

  2. To modify JUPITR's MODPARAMS.DAT when logged in on any other computer that boots from the same common system disk, enter the following command:
    $ EDIT SYS$SYSDEVICE:[SYS0.SYSEXE]MODPARAMS.DAT
  3. To modify records in the cluster common system authorization file in a cluster with a single, cluster-common system disk, enter the following commands on any computer:
    $ SET DEFAULT SYS$COMMON:[SYSEXE]$ RUN SYS$SYSTEM:AUTHORIZE
  4. To modify records in a computer-specific system authorization file when logged in to another computer that boots from the same cluster common system disk, you must set your default directory to the specific computer. For example, if you have set up a computer-specific system authorization file(SYSUAF.DAT) for computer JUPITR, you must set your default directory to JUPITR's computer-specific [SYSEXE] directory before invoking AUTHORIZE, as follows:
    $ SET DEFAULT SYS$SYSDEVICE:[SYS0.SYSEXE]
    $ RUN SYS$SYSTEM:AUTHORIZE

5.4. Clusterwide Logical Names

Clusterwide logical names, introduced in OpenVMS Version 7.2, extend the convenience and ease-of-use features of shareable logical names to OpenVMS Cluster systems. Clusterwide logical names are available on OpenVMS Integrity servers and OpenVMS Alpha systems, in a single or a mixed architecture OpenVMS Cluster.

Existing applications can take advantage of clusterwide logical names without any changes to the application code. Only a minor modification to the logical name tables referenced by the application (directly or indirectly) is required.

New logical names are local by default. Clusterwide is an attribute of a logical name table. In order for a new logical name to be clusterwide, it must be created in a clusterwide logical name table.

Some of the most important features of clusterwide logical names are:
  • When a new node joins the cluster, it automatically receives the current set of clusterwide logical names.

  • When a clusterwide logical name or name table is created, modified, or deleted, the change is automatically propagated to every other node in the cluster running OpenVMS Version 7.2 or later. Modifications include security profile changes to a clusterwide table.

  • Translations are done locally so there is minimal performance degradation for clusterwide name translations.

  • Because LNM$CLUSTER_TABLE and LNM$SYSCLUSTER_TABLE exist on all systems running OpenVMS Version 7.2 or later, the programs and command procedures that use clusterwide logical names can be developed, tested, and run on nonclustered systems.

5.4.1. Default Clusterwide Logical Name Tables

To support clusterwide logical names, the operating system creates two clusterwide logical name tables and their logical names at system startup, as shown in Table 5.1, “Default Clusterwide Logical Name Tables and Logical Names”. These logical name tables and logical names are in addition to the ones supplied for the process, job, group, and system logical name tables. The names of the clusterwide logical name tables are contained in the system logical name directory, LNM$SYSTEM_DIRECTORY.
Table 5.1. Default Clusterwide Logical Name Tables and Logical Names

Name

Purpose

LNM$SYSCLUSTER_TABLE

The default table for clusterwide system logical names. It is empty when shipped. This table is provided for system managers who want to use clusterwide logical names to customize their environments. The names in this table are available to anyone translating a logical name using SHOW LOGICAL/SYSTEM, specifying a table name of LNM$SYSTEM, or LNM$DCL_LOGICAL (DCL's default table search list), or LNM$FILE_DEV (system and RMS default).

LNM$SYSCLUSTER

The logical name for LNM$SYSCLUSTER_TABLE. It is provided for convenience in referencing LNM$SYSCLUSTER_TABLE. It is consistent in format with LNM$SYSTEM_TABLE and its logical name, LNM$SYSTEM.

LNM$CLUSTER_TABLE

The parent table for all clusterwide logical name tables, including LNM$SYSCLUSTER_TABLE. When you create a new table using LNM$CLUSTER_TABLE as the parent table, the new table will be available clusterwide.

LNM$CLUSTER

The logical name for LNM$CLUSTER_TABLE. It is provided for convenience in referencing LNM$CLUSTER_TABLE.

5.4.2. Translation Order

The definition of LNM$SYSTEM has been expanded to include LNM$SYSCLUSTER. When a system logical name is translated, the search order is LNM$SYSTEM_TABLE, LNM$SYSCLUSTER_TABLE. Because the definitions for the system default table names, LNM$FILE_DEV and LNM$DCL_LOGICALS, include LNM$SYSTEM, translations using those default tables include definitions in LNM$SYSCLUSTER.

The current precedence order for resolving logical names is preserved. Clusterwide logical names that are translated against LNM$FILE_DEV are resolved last, after system logical names. The precedence order, from first to last, is process → job → group → system → cluster, as shown in Figure 5.5, “Translation Order Specified by LNM$FILE_DEV”.

Figure 5.5. Translation Order Specified by LNM$FILE_DEV
Translation Order Specified by LNM$FILE_DEV

5.4.3. Creating Clusterwide Logical Name Tables

You might want to create additional clusterwide logical name tables for the following purposes:
  • For a multiprocess clusterwide application to use

  • For members of a UIC group to share

To create a clusterwide logical name table, you must have create (C) access to the parent table and write (W) access to LNM$SYSTEM_DIRECTORY, or the SYSPRV(system) privilege.

A shareable logical name table has UIC-based protection. Each class of user(system (S), owner (O), group (G), and world (W)) can be granted four types of access: read (R), write (W), create (C), or delete (D).

You can create additional clusterwide logical name tables in the same way that you can create additional process, job, and group logical name tables – with the CREATE/NAME_TABLE command or with the $CRELNT system service. When creating a clusterwide logical name table, you must specify the /PARENT_TABLE qualifier and provide a value for the qualifier that is a clusterwide table name. Any existing clusterwide table used as the parent table will make the new table clusterwide.

The following example shows how to create a clusterwide logical name table:
$ CREATE/NAME_TABLE/PARENT_TABLE=LNM$CLUSTER_TABLE -
_$ new-clusterwide-logical-name-table

5.4.4. Alias Collisions Involving Clusterwide Logical Name Tables

Alias collisions involving clusterwide logical name tables are treated differently from alias collisions of other types of logical name tables. Table 5.2, “Alias Collisions and Outcomes ” describes the types of collisions and their outcomes.
Table 5.2. Alias Collisions and Outcomes

Collision Type

Outcome

Creating a local table with same name and access mode as an existing clusterwide table

New local table is not created. The condition value SS$_NORMAL is returned, which means that the service completed successfully but the logical name table already exists. The existing clusterwide table and its names on all nodes remain in effect.

Creating a clusterwide table with same name and access mode as an existing local table

New clusterwide table is created. The condition value SS$_LNMCREATED is returned, which means that the logical name table was created. The local table and its names are deleted. If the clusterwide table was created with the DCL command DEFINE, a message is displayed:
DCL-I-TABSUPER, previous table table_name
has been superseded

If the clusterwide table was created with the $CRELNT system service,$CRELNT returns the condition value: SS$_SUPERSEDE.

Creating a clusterwide table with same name and access mode as an existing clusterwide table

New clusterwide table is not created. The condition value SS$_NORMAL is returned, which means that the service completed successfully but the logical name table already exists. The existing table and all its names remain in effect, regardless of the setting of the$CRELNT system service's CREATE-IF attribute. This prevents surprise implicit deletions of existing table names from other nodes.

5.4.5. Creating Clusterwide Logical Names

To create a clusterwide logical name, you must have write (W) access to the table in which the logical name is to be entered, or SYSNAM privilege if you are creating clusterwide logical names only in LNM$SYSCLUSTER. Unless you specify an access mode (user, supervisor, and so on), the access mode of the logical name you create defaults to the access mode from which the name was created. If you created the name with a DCL command, the access mode defaults to supervisor mode. If you created the name with a program, the access mode typically defaults to user mode.

When you create a clusterwide logical name, you must include the name of a clusterwide logical name table in the definition of the logical name. You can create clusterwide logical names by using DCL commands or with the $CRELNM system service.

The following example shows how to create a clusterwide logical name in the default clusterwide logical name table, LNM$CLUSTER_TABLE, using the DEFINE command:
$ DEFINE/TABLE=LNM$CLUSTER_TABLE logical-name equivalence-string
To create clusterwide logical names that will reside in a clusterwide logical name table you created, you define the new clusterwide logical name with the DEFINE command, specifying your new clusterwide table's name with the /TABLE qualifier, as shown in the following example:
$ DEFINE/TABLE=new-clusterwide-logical-name-table logical-name -
_$ equivalence-string

Note

If you attempt to create a new clusterwide logical name with the same access mode and identical equivalence names and attributes as an existing clusterwide logical name, the existing name is not deleted, and no messages are sent to remote nodes. This behavior differs from similar attempts for other types of logical names, which delete the existing name and create the new one. For clusterwide logical names, this difference is a performance enhancement.

The condition value SS$_NORMAL is returned. The service completed successfully, but the new logical name was not created.

5.4.6. Management Guidelines

When using clusterwide logical names, observe the following guidelines:
  1. Do not use certain logical names clusterwide.

    he following logical names are not valid for clusterwide use:
    • Mailbox names, because mailbox devices are local to a node.

    • SYS$NODE and SYS$NODE_FULLNAME must be in LNM$SYSTEM_TABLE and are node specific.

    • LMF$LICENSE_TABLE.

  2. Do not redefine LNM$SYSTEM.

    LNM$SYSTEM is now defined as LNM$SYSTEM_TABLE, LNM$SYSCLUSTER_TABLE. Do not reverse the order of these two tables. If you do, then any names created using the /SYSTEM qualifier or in LNM$SYSTEM would go in LNM$SYSCLUSTER_TABLE and be clusterwide. Various system failures would result. For example, the MOUNT/SYSTEM command would attempt to create a clusterwide logical name for a mounted volume, which would result in an error.

  3. Keep LNM$SYSTEM contents in LNM$SYSTEM.

    Do not merge the logical names in LNM$SYSTEM into LNM$SYSCLUSTER. Many system logical names in LNM$SYSTEM contain system roots and either node-specific devices, or node-specific directories, or both.

  4. Adopt naming conventions for logical names used at your site.

    To avoid confusion and name conflicts, develop one naming convention for system-specific logical names and another for clusterwide logical names.

  5. Avoid using the dollar sign ($) in your own site's logical names, because OpenVMS software uses it in its names.

  6. Be aware that clusterwide logical name operations will stall when the clusterwide logical name database is not consistent.

    This can occur during system initialization when the system's clusterwide logical name database is not completely initialized. It can also occur when the cluster server process has not finished updating the clusterwide logical name database, or during resynchronization after nodes enter or leave the cluster. As soon as consistency is reestablished, the processing of clusterwide logical name operations resumes.

5.4.7. Using Clusterwide Logical Names in Applications

The $TRNLNM system service and the $GETSYI system service provide attributes that are specific to clusterwide logical names. This section describes those attributes. It also describes the use of$CRELNT as it pertains to creating a clusterwide table. For more information about using logical names in applications, refer to the VSI OpenVMS Programming Concepts Manual.

5.4.7.1. Clusterwide Attributes for $TRNLNM System Service

Two clusterwide attributes are available in the $TRNLNM system service:
  • LNM$V_CLUSTERWIDE

  • LNM$M_INTERLOCKED

LNM$V_CLUSTERWIDE is an output attribute to be returned in the item list if you asked for the LNM$_ATTRIBUTES item for a logical name that is clusterwide.

LNM$M_INTERLOCKED is an attr argument bit that can be set to ensure that any clusterwide logical name modifications in progress are completed before the name is translated. LNM$M_INTERLOCKED is not set by default. If your application requires translation using the most recent definition of a clusterwide logical name, use this attribute to ensure that the translation is stalled until all pending modifications have been made.

On a single system, when one process modifies the shareable part of the logical name database, the change is visible immediately to other processes on that node. Moreover, while the modification is in progress, no other process can translate or modify shareable logical names.

In contrast, when one process modifies the clusterwide logical name database, the change is visible immediately on that node, but it takes a short time for the change to be propagated to other nodes. By default, translations of clusterwide logical names are not stalled. Therefore, it is possible for processes on different nodes to translate a logical name and get different equivalence names when modifications are in progress.

The use of LNM$M_INTERLOCKED guarantees that your application will receive the most recent definition of a clusterwide logical name.

5.4.7.2. Clusterwide Attribute for $GETSYI System Service

The clusterwide attribute, SYI$_CWLOGICALS, has been added to the $GETSYI system service. When you specify SYI$_CWLOGICALS, $GETSYI returns the value 1 if the clusterwide logical name database has been initialized on the CPU, or the value 0 if it has not been initialized. Because this number is a Boolean value (1 or 0), the buffer length field in the item descriptor should specify 1 (byte). On a nonclustered system, the value of SYI$_CWLOGICALS is always 0.

5.4.7.3. Creating Clusterwide Tables with the $CRELNT System Service

When creating a clusterwide table, the $CRELNT requester must supply a table name. OpenVMS does not supply a default name for clusterwide tables because the use of default names enables a process without the SYSPRV privilege to create a shareable table.

5.4.8. Defining and Accessing Clusterwide Logical Names

Initializing the clusterwide logical name database on a booting node requires sending a message to another node and having its CLUSTER_SERVER process reply with one or messages containing a description of the database. The CLUSTER_SERVER process on the booting node requests system services to create the equivalent names and tables. How long this initialization takes varies with conditions such as the size of the clusterwide logical name database, the speed of the cluster interconnect, and the responsiveness of the CLUSTER_SERVER process on the responding node.

Until a booting node's copy of the clusterwide logical name database is consistent with the logical name databases of the rest of the cluster, any attempt on the booting node to create or delete clusterwide names or tables is stalled transparently. Because translations are not stalled by default, any attempt to translate a clusterwide name before the database is consistent may fail or succeed, depending on timing. To stall a translation until the database is consistent, specify the F$TRNLNM CASE argument as INTERLOCKED.

5.4.8.1. Defining Clusterwide Logical Names in SYSTARTUP_VMS.COM

In general, system managers edit the SYLOGICALS.COM command procedure to define site-specific logical names that take effect at system startup. However, VSI recommends that, if possible, clusterwide logical names be defined in the SYSTARTUP_VMS.COM command procedure instead with the exception of those logical names discussed in Section 5.4.8.2, “Defining Certain Logical Names in SYLOGICALS.COM”. The reason for defining clusterwide logical names in SYSTARTUP_VMS.COM is that SYSTARTUP_VMS.COM is run at a much later stage in the booting process than SYLOGICALS.COM.

OpenVMS startup is single streamed and synchronous except for actions taken by created processes, such as the CLUSTER_SERVER process. Although the CLUSTER_SERVER process is created very early in startup, it is possible that when SYLOGICALS.COM is executed, the booting node's copy of the clusterwide logical name database has not been fully initialized. In such a case, a clusterwide definition in SYLOGICALS.COM would stall startup and increase the time it takes for the system to become operational.

OpenVMS will ensure that the clusterwide database has been initialized before SYSTARTUP_VMS.COM is executed.

5.4.8.2. Defining Certain Logical Names in SYLOGICALS.COM

To be effective, certain logical names, such as LMF$LICENSE, NET$PROXY, and VMS$OBJECTS must be defined earlier in startup than when SYSTARTUP_VMS.COM is invoked. Most such names are defined in SYLOGICALS.COM, with the exception of VMS$OBJECTS, which is defined in SYSECURITY.COM, and any names defined in SYCONFIG.COM.

Although VSI recommends defining clusterwide logical names in SYSTARTUP_VMS.COM, to define these names to be clusterwide, you must do so in SYLOGICALS.COM or SYSECURITY.COM. Note that doing this may increase startup time.

Alternatively, you can take the traditional approach and define these names as systemwide logical names with the same definition on every node.

5.4.8.3. Using Conditional Definitions for Startup Command Procedures

For clusterwide definitions in any startup command procedure that is common to all cluster nodes, VSI recommends that you use a conditional definition. For example:
$ IF F$TRNLNM("CLUSTER_APPS") .EQS. "" THEN -
_$ DEFINE/TABLE=LNM$SYSCLUSTER/EXEC CLUSTER_APPS -
_$ $1$DKA500:[COMMON_APPS]

A conditional definition can prevent unpleasant surprises. For example, suppose a system manager redefines a name that is also defined in SYSTARTUP_VMS.COM but does not edit SYSTARTUP_VMS.COM because the new definition is temporary. If a new node joins the cluster, the new node would initially receive the new definition. However, when the new node executes SYSTARTUP_VMS.COM, it will cause all the nodes in the cluster, including itself, to revert to the original value.

If you include a conditional definition in SYLOGICALS.COM or SYSECURITY.COM, specify the F$TRNLNM CASE argument as INTERLOCKED to ensure that clusterwide logical names have been fully initialized before the translation completes. An example of a conditional definition with the argument specified follows:
$ IF F$TRNLNM("CLUSTER_APPS",,,,"INTERLOCKED") .EQS.
"" THEN - _$ DEFINE/TABLE=LNM$SYSCLUSTER/EXEC CLUSTER_APPS -
_$ $1$DKA500:[COMMON_APPS]

Note

F$GETSYI ("CWLOGICALS") always returns a value of FALSE on a noncluster system. Procedures that are designed to run in both clustered and nonclustered environments should first determine whether they are in a cluster and, if so, then determine whether clusterwide logical names are initialized.

5.4.9. Displaying Clusterwide Logical Names

The /CLUSTER qualifier was added to the SHOW LOGICAL DCL command in OpenVMS Version 8.2. When the SHOW LOGICAL/CLUSTER command is specified, all clusterwide logical names are displayed, as shown in the following example:
$ SHOW LOGICAL/CLUSTER

(LNM$CLUSTER_TABLE)

(LNM$SYSCLUSTER_TABLE)

  "MSCPMOUNT$_AMALFI_LAST" = "2005-10-10 14:25:03.74"
  "MSCPMOUNT$_AMALFI_LOGINTIM" = " 8-OCT-2005 01:02:22.17"
  "MSCPMOUNT$_AMALFI_NEXT" = "2005-10-10 14:40:03.74"
  "MSCPMOUNT$_AMALFI_PID" = "26200462"
  .
  .
  .
  "MSCPMOUNT$_ETNA_LAST" = "2005-10-10 14:25:18.78"
  "MSCPMOUNT$_ETNA_LOGINTIM" = " 8-OCT-2005 07:44:37.89"
  "MSCPMOUNT$_ETNA_NEXT" = "2005-10-10 14:40:18.79"
  "MSCPMOUNT$_ETNA_PID" = "26A0044E"
  .
  .
  .
  "MSCPMOUNT$_MILAN_LAST" = "2005-10-10 14:25:19.64"
  "MSCPMOUNT$_MILAN_LOGINTIM" = " 8-OCT-2005 07:22:08.05"
  "MSCPMOUNT$_MILAN_NEXT" = "2005-10-10 14:40:19.64"
  "MSCPMOUNT$_MILAN_PID" = "26600458"
  .
  .
  .
  "MSCPMOUNT$_ORVIET_LAST" = "2005-10-10 14:29:25.94"
  "MSCPMOUNT$_ORVIET_LOGINTIM" = "30-SEP-2005 09:38:27.38"
  "MSCPMOUNT$_ORVIET_NEXT" = "2005-10-10 14:44:26.61"
  "MSCPMOUNT$_ORVIET_PID" = "25600139"
  .
  .
  .
  "MSCPMOUNT$_TURIN_LAST" = "2005-10-10 14:39:59.59"
  "MSCPMOUNT$_TURIN_LOGINTIM" = "10-OCT-2005 09:22:48.46"
  "MSCPMOUNT$_TURIN_NEXT" = "2005-10-10 14:54:59.59"
  "MSCPMOUNT$_TURIN_PID" = "2760012C"
  "PREPOPULATE_NEXT_STREAM$IGNORE_BUILD_MASTER_944" = "1"

(CLU$ICC_ORBS_AMALFI)

  "ICC$ORB_ICC$PID_26200450_U" = "T"
      = "M\.v....k...............æ...æ...þ...þ.....AMALFI::ICC$PID_26200450_U....."
  "ICC$ORB_REG$SERVER_E" = "T"
      = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_E044........"
  "ICC$ORB_REG$SERVER_K" = "T"
      = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_K044........"
  "ICC$ORB_REG$SERVER_U" = "T"
      = "p.O<....e...............æ...æ...þ...þ.....AMALFI::REG$SERVER_U044........"

(CLU$ICC_ORBS_ETNA)

(CLU$ICC_ORBS_MILAN)

(CLU$ICC_ORBS_ORVIET)

  "ICC$ORB_ICC$PID_26000450_U" = "T"
      = "VQ.p....k...............æ...æ...þ...þ.....ETNA::ICC$PID_26000450_U......."

(CLU$ICC_ORBS_TURIN)

.
.
.
(ICC$REGISTRY_TABLE)

5.5. Coordinating Startup Command Procedures

Immediately after a computer boots, it runs the site-independent command procedure SYS$SYSTEM:STARTUP.COM to start up the system and control the sequence of startup events. The STARTUP.COM procedure calls a number of other startup command procedures that perform cluster-specific and node-specific tasks.

The following sections describe how, by setting up appropriate cluster-specific startup command procedures and other system files, you can prepare the OpenVMS Cluster operating environment on the first installed computer before adding other computers to the cluster.

Reference: See also the VSI OpenVMS System Manager's Manual for more information about startup command procedures.

5.5.1. OpenVMS Startup Procedures

Several startup command procedures are distributed as part of the OpenVMS operating system. The SYS$SYSTEM:STARTUP.COM command procedure executes immediately after OpenVMS is booted and invokes the site-specific startup command procedures described in the following table.
Procedure NameInvoked byFunction
  • SYS$MANAGER:
  • YPAGSWPFILES.COM
  • SYS$SYSTEM:
  • STARTUP.COM

A file to which you add commands to install page and swap files (other than the primary page and swap files that are installed automatically).

  • SYS$MANAGER:
  • SYCONFIG.COM
  • SYS$SYSTEM:
  • STARTUP.COM

Connects special devices and loads device I/O drivers.

  • SYS$MANAGER:
  • SYSECURITY.COM
  • SYS$SYSTEM:
  • STARTUP.COM

Defines the location of the security audit and archive files before it starts the security audit server.

  • SYS$MANAGER:
  • SYLOGICALS.COM
  • SYS$SYSTEM:
  • STARTUP.COM

Creates systemwide logical names, and defines system components as executive-mode logical names. (Clusterwide logical names should be defined in SYSTARTUP_VMS.COM.) Cluster common disks can be mounted at the end of this procedure.

  • SYS$MANAGER:
  • SYSTARTUP_VMS.COM
  • SYS$SYSTEM:
  • STARTUP.COM
Performs many of the following startup and login functions:
  • Mounts all volumes except the system disk.

  • Sets device characteristics.

  • Defines clusterwide logical names

  • Initializes and starts batch and print queues.

  • Installs known images.

  • Starts layered products.

  • Starts the DECnet software.

  • Analyzes most recent system failure.

  • Purges old operator log files.

  • Starts the LAT network (if used).

  • Defines the maximum number of interactive users.

  • Announces that the system is up and running.

  • Allows users to log in.

The directory SYS$COMMON:[SYSMGR] contains a template file for each command procedure that you can edit. Use the command procedure templates (in SYS$COMMON:[SYSMGR]*.TEMPLATE) as examples for customization of your system's startup and login characteristics.

5.5.2. Building Startup Procedures

The first step in preparing an OpenVMS Cluster shared environment is to build a SYSTARTUP_VMS command procedure. Each computer executes the procedure at startup time to define the operating environment.

Prepare the SYSTARTUP_VMS.COM procedure as follows:
StepAction

1

In each computer's SYS$SPECIFIC:[SYSMGR] directory, edit the SYSTARTUP_VMS.TEMPLATE file to set up a SYSTARTUP_VMS.COM procedure that:
  • Performs computer-specific startup functions such as the following:
    • Setting up dual-ported and local disks

    • Loading device drivers

    • Setting up local terminals and terminal server access

  • Invoking the common startup procedure (described next).

2

Build a common command procedure that includes startup commands that you want to be common to all computers. The common procedure might contain commands that:
  • Install images

  • Define logical names

  • Set up queues

  • Set up and mount physically accessible mass storage devices

  • Perform any other common startup functions

Note: You might choose to build these commands into individual command procedures that are invoked from the common procedure. For example, the MSCPMOUNT.COM file in the SYS$EXAMPLES directory is a sample common command procedure that contains commands typically used to mount cluster disks. The example includes comments explaining each phase of the procedure.

3

Place the common procedure in the SYS$COMMON:[SYSMGR] directory on a common system disk or other cluster-accessible disk.

Important: The common procedure is usually located in the SYS$COMMON:[SYSMGR] directory on a common system disk but can reside on any disk, provided that the disk is cluster accessible and is mounted when the procedure is invoked. If you create a copy of the common procedure for each computer, you must remember to update each copy whenever you make changes.

5.5.3. Combining Existing Procedures

To build startup procedures for an OpenVMS Cluster system in which existing computers are to be combined, you should compare both the computer-specific SYSTARTUP_VMS and the common startup command procedures on each computer and make any adjustments required. For example, you can compare the procedures from each computer and include commands that define the same logical names in your common SYSTARTUP_VMS command procedure.

After you have chosen which commands to make common, you can build the common procedures on one of the OpenVMS Cluster computers.

5.5.4. Using Multiple Startup Procedures

To define a multiple-environment cluster, you set up computer-specific versions of one or more system files. For example, if you want to give users larger working set quotas on URANUS, you would create a computer-specific version of SYSUAF.DAT and place that file in system's root directory. That directory can be located in URANUS's root on a common system disk or on an individual system disk that you have set up on URANUS.

Follow these steps to build SYSTARTUP and SYLOGIN command files for a multiple-environment OpenVMS Cluster:

Step

Action

1

Include in SYSTARTUP_VMS.COM elements that you want to remain unique to a computer, such as commands to define computer-specific logical names and symbols.

2

Place these files in the SYS$SPECIFIC root on each computer.

Example: Consider a three-member cluster consisting of computers JUPITR, SATURN, and PLUTO. The time sharing environments on JUPITR and SATURN are the same. However, PLUTO runs applications for a specific user group. In this cluster, you would create a common SYSTARTUP_VMS command procedure for JUPITR and SATURN that defines identical environments on these computers. But the command procedure for PLUTO would be different; it would include commands to define PLUTO's special application environment.

5.6. Providing OpenVMS Cluster System Security

The OpenVMS security subsystem ensures that all authorization information and object security profiles are consistent across all nodes in the cluster. The OpenVMS operating system does not support multiple security domains because the operating system cannot enforce a level of separation needed to support different security domains on separate cluster members.

5.6.1. Security Checks

In an OpenVMS Cluster system, individual nodes use a common set of authorizations to mediate access control that, in effect, ensures that a security check results in the same answer from any node in the cluster. The following list outlines how the OpenVMS operating system provides a basic level of protection:
  • Authorized users can have processes executing on any OpenVMS Cluster member.

  • A process, acting on behalf of an authorized individual, requests access to a cluster object.

  • A coordinating node determines the outcome by comparing its copy of the common authorization database with the security profile for the object being accessed.

The OpenVMS operating system provides the same strategy for the protection of files and queues, and further incorporates all other cluster-visible objects, such as devices, volumes, and lock resource domains.

Starting with OpenVMS Version 7.3, the operating system provides clusterwide intrusion detection, which extends protection against attacks of all types throughout the cluster. The intrusion data and information from each system is integrated to protect the cluster as a whole. Prior to Version 7.3, each system was protected individually.

The SECURITY_POLICY system parameter controls whether a local or a clusterwide intrusion database is maintained for each system. The default setting is for a clusterwide database, which contains all unauthorized attempts and the state of any intrusion events for all cluster members that are using this setting. Cluster members using the clusterwide intrusion database are made aware if a cluster member is under attack or has any intrusion events recorded. Events recorded on one system can cause another system in the cluster to take restrictive action. (For example, the person attempting to log in is monitored more closely and limited to a certain number of login retries within a limited period of time. Once a person exceeds either the retry or time limitation, he or she cannot log in).

Actions of the cluster manager in setting up an OpenVMS Cluster system can affect the security operations of the system. You can facilitate OpenVMS Cluster security management using the suggestions discussed in the following sections.

The easiest way to ensure a single security domain is to maintain a single copy of each of the following files on one or more disks that are accessible from anywhere in the OpenVMS Cluster system. When a cluster is configured with multiple system disks, you can use system logical names (as shown in Section 5.8, “Coordinating System Files”) to ensure that only a single copy of each file exists.

  • SYS$MANAGER:VMS$AUDIT_SERVER.DAT
  • SYS$SYSTEM:NETOBJECT.DAT
  • SYS$SYSTEM:NETPROXY.DAT
  • TCPIP$PROXY.DAT
  • SYS$SYSTEM:PE$IP_CONFIG.DAT
  • SYS$SYSTEM:QMAN$MASTER.DAT
  • SYS$SYSTEM:RIGHTSLIST.DAT
  • SYS$SYSTEM:SYSALF.DAT
  • SYS$SYSTEM:SYSUAF.DAT
  • SYS$SYSTEM:SYSUAFALT.DAT
  • SYS$SYSTEM:VMS$PASSWORD_HISTORY.DATA
  • SYS$SYSTEM:VMSMAIL_PROFILE.DATA
  • SYS$LIBRARY:VMS$PASSWORD_DICTIONARY.DATA
  • SYS$LIBRARY:VMS$PASSWORD_POLICY.EXE

Note: Using shared files is not the only way of achieving a single security domain. You may need to use multiple copies of one or more of these files on different nodes in a cluster. For example, on Alpha nodes you may choose to deploy system-specific user authorization files (SYSUAFs) to allow for different memory management working-set quotas among different nodes. Such configurations are fully supported as long as the security information available to each node in the cluster is identical.

5.6.2. Files Relevant to OpenVMS Cluster Security

Table 5.3, “Security Files” describes the security-relevant portions of the files that must be common across all cluster members to ensure that a single security domain exists.

Notes:
  • Some of these files are created only on request and may not exist in all configurations.

  • A file can be absent on one node only if it is absent on all nodes.

  • As soon as a required file is created on one node, it must be created or commonly referenced on all remaining cluster nodes.

The following table describes designations for the files in Table 5.3, “Security Files”.

Table Keyword

Meaning

Required

The file contains some data that must be kept common across all cluster members to ensure that a single security environment exists.

Recommended

The file contains data that should be kept common at the discretion of the site security administrator or system manager. Nonetheless, VSI recommends that you synchronize the recommended files.


Table 5.3. Security Files
File NameContains

CLUSTER_AUTHORIZE.DAT

The cluster authorization file, SYS$COMMON:[SYSEXE]CLUSTER_AUTHORIZE.DAT, contains the cluster group number in a disorderly form and the cluster password. The CLUSTER_AUTHORIZE.DAT file is accessible only to users with the SYSPRV privilege.

PE$IP_CONFIG.DAT [recommended]

For cluster over IP configurations, which are using IP unicast, the remote node IP address should be present in the existing cluster members file in the SYS$SYSTEM:PE$IP_CONFIG.DAT file. Remote nodes in a different IP multicast domain can use the IP unicast messaging technique to join the Cluster.

VMS$AUDIT_SERVER.DAT [recommended]

Information related to security auditing. Among the information contained is the list of enabled security auditing events and the destination of the system security audit journal file. When more than one copy of this file exists, all copies should be update dafter any SET AUDIT command.

OpenVMS Cluster system managers should ensure that the name assigned to the security audit journal file resolves to the following location:
SYS$COMMON:[SYSMGR]SECURITY.AUDIT$JOURNAL

Rule: If you need to relocate the audit journal file somewhere other than the system disk (or if you have multiple system disks), you should redirect the audit journal uniformly across all nodes in the cluster. Use the command SETAUDIT/JOURNAL=SECURITY/DESTINATION= file-name, specifying a file name that resolves to the same file throughout the cluster.

Changes are automatically made in the audit server database, SYS$MANAGER:VMS$AUDIT_SERVER.DAT. This database also identifies which events are enabled and how to monitor the audit system's use of resources, and restores audit system settings each time the system is rebooted.

Caution: Failure to synchronize multiple copies of this file properly may result in partitioned auditing domains.

Reference: For more information, see the VSI OpenVMS Guide to System Security.

NETOBJECT.DAT [required]

The DECnet object database. Among the information contained in this file is the list of known DECnet server accounts and passwords. When more than one copy of this file exists, all copies must be updated after every use of the NCP commands SET OBJECT or DEFINE OBJECT.

Caution: Failure to synchronize multiple copies of this file properly may result in unexplained network login failures and unauthorized network access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

Reference: Refer to the DECnet–Plus documentation for equivalent NCL command information.

NETPROXY.DAT and NET$PROXY.DAT [required]

The network proxy database. It is maintained by the OpenVMS Authorize utility. When more than one copy of this file exists, all copies must be updated after any UAF proxy command.

Note: The NET$PROXY.DAT and NETPROXY.DAT files are equivalent;NET$PROXY.DAT is for DECnet–Plus implementations and NETPROXY.DAT is for DECnet for OpenVMS implementations.

Caution: Failure to synchronize multiple copies of this file properly may result in unexplained network login failures and unauthorized network access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

Reference: Appendix B, Building Common Files discusses how to consolidate several NETPROXY.DAT and RIGHTSLIST.DAT files.

TCPIP$PROXY.DAT

This database provides OpenVMS identities for remote NFS clients and UNIX-style identifiers for local NFS client users;provides proxy accounts for remote processes. For more information about this file, see the VSI TCP/IP Services for OpenVMS Management.

QMAN$MASTER.DAT [required]

The master queue manager database. This file contains the security information for all shared batch and print queues.

Rule: If two or more nodes are to participate in a shared queuing system, a single copy of this file must be maintained on a shared disk. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

RIGHTSLIST.DAT [required]

The rights identifier database. It is maintained by the OpenVMS Authorize utility and by various rights identifier system services. When more than one copy of this file exists, all copies must be updated after any change to any identifier or holder records.

Caution: Failure to synchronize multiple copies of this file properly may result in unauthorized system access and unauthorized access to protected objects. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

Reference: Appendix B, Building Common Files discusses how to consolidate several NETPROXY.DAT and RIGHTSLIST.DAT files.

SYSALF.DAT [required]

The system Auto login facility database. It is maintained by the OpenVMS SYSMAN utility. When more than one copy of this file exists, all copies must be updated after any SYSMAN ALF command.

Note: This file may not exist in all configurations.

Caution: Failure to synchronize multiple copies of this file properly may result in unexplained login failures and unauthorized system access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

SYSUAF.DAT [required]

The system user authorization file. It is maintained by the OpenVMS Authorize utility and is modifiable via the$SETUAI system service. When more than one copy of this file exists, you must ensure that the SYSUAF and associated $SETUAI item codes are synchronized for each user record. The following table shows the fields in SYSUAF and their associated $SETUAI item codes.

Internal Field Name

$SETUAI Item Code

UAF$R_DEF_CLASS

UAI$_DEF_CLASS

UAF$Q_DEF_PRIV

UAI$_DEF_PRIV

UAF$B_DIALUP_ACCESS_P

UAI$_LOCAL_ACCESS_P

UAF$B_LOCAL_ACCESS_S

UAI$_DIALUP_ACCESS_S

UAF$B_ENCRYPT

UAI$_ENCRYPT

UAF$B_ENCRYPT2

UAI$_ENCRYPT2

UAF$Q_EXPIRATION

UAI$_EXPIRATION

UAF$L_FLAGS

UAI$_FLAGS

UAF$B_LOCAL_ACCESS_P

UAI$_LOCAL_ACCESS_P

UAF$B_LOCAL_ACCESS_S

UAI$_LOCAL_ACCESS_S

UAF$B_NETWORK_ACCESS_P

UAI$_NETWORK_ACCESS_P

UAF$B_NETWORK_ACCESS_S

UAI$_NETWORK_ACCESS_S

UAF$B_PRIME_DAYS

UAI$_PRIMEDAYS

UAF$Q_PRIV

UAI$_PRIV

UAF$Q_PWD

UAI$_PWD

UAF$Q_PWD2

UAI$_PWD2

UAF$Q_PWD_DATE

UAI$_PWD_DATE

UAF$Q_PWD2_DATE

UAI$_PWD2_DATE

UAF$B_PWD_LENGTH

UAI$_PWD_LENGTH

UAF$Q_PWD_LIFETIME

UAI$_PWD_LIFETIME

UAF$B_REMOTE_ACCESS_P

UAI$_REMOTE_ACCESS_P

UAF$B_REMOTE_ACCESS_S

UAI$_REMOTE_ACCESS_S

UAF$R_MAX_CLASS

UAI$_MAX_CLASS

UAF$R_MIN_CLASS

UAI$_MIN_CLASS

UAF$W_SALT

UAI$_SALT

UAF$L_UIC

Not applicable

Caution: Failure to synchronize multiple copies of the SYSUAF files properly may result in unexplained login failures and unauthorized system access. For instructions on maintaining a single copy, refer to Section 5.8.1, “Procedure”.

Reference: Appendix B, Building Common Files discusses creation and management of the various elements of an OpenVMS Cluster common SYSUAF.DAT authorization database.

SYSUAFALT.DAT [required]

The system alternate user authorization file. This file serves as a backup to SYSUAF.DAT and is enabled via the SYSUAFALT system parameter. When more than one copy of this file exists, all copies must be updated after any change to any authorization records in this file.

Note: This file may not exist in all configurations.

Caution: Failure to synchronize multiple copies of this file properly may result in unexplained login failures and unauthorized system access.

VMS$PASSWORD_HISTORY.DATA [recommended]

The system password history database. It is maintained by the system password change facility. When more than one copy of this file exists, all copies should be updated after any password change.

Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy.

VMSMAIL_PROFILE.DATA [recommended]

The system mail database. This file is maintained by the OpenVMS Mail utility and contains mail profiles for all system users. Among the information contained in this file is the list of all mail forwarding addresses in use on the system. When more than one copy of this file exists, all copies should be updated after any changes to mail forwarding.

Caution: Failure to synchronize multiple copies of this file properly may result in unauthorized disclosure of information.

VMS$PASSWORD_DICTIONARY.DATA [recommended]

The system password dictionary. The system password dictionary is a list of English language words and phrases that are not legal for use as account passwords. When more than one copy of this file exists, all copies should be updated after any site-specific additions.

Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy.

VMS$PASSWORD_POLICY.EXE [recommended]

Any site-specific password filters. It is created and installed by the site-security administrator or system manager. When more than one copy of this file exists, all copies should be identical.

Caution: Failure to synchronize multiple copies of this file properly may result in a violation of the system password policy.

Note: System managers can create this file as an image to enforce their local password policy. This is an architecture-specific image file that cannot be shared among different architecture types.

5.7. Network Security

Network security must promote interoperability and uniform security approaches throughout networks. The following list shows three major areas of network security:
  • User authentication

    On Cluster systems connected using IP, ensure that the cluster communications over insecure WAN links are encrypted and authenticated.

  • OpenVMS Cluster membership management

    On Cluster systems connected using IP, isolate IP subnets that are used for cluster communication from the public internet using a secure gateway as shown in Figure 5.6, “Virtual Private Network for Protecting Cluster Traffic”.

    Figure 5.6. Virtual Private Network for Protecting Cluster Traffic
    Virtual Private Network for Protecting Cluster Traffic
  • Using a security audit log file

OpenVMS Cluster system managers must also ensure consistency in the use of DECnet software for intracluster communication.

5.7.1. Mechanisms

Depending on the level of network security required, you might also want to consider how other security mechanisms, such as protocol encryption and decryption, can promote additional security protection across the cluster.

Reference: See the VSI OpenVMS Guide to System Security.

5.8. Coordinating System Files

Follow these guidelines to coordinate system files:

IF you are setting up...

THEN follow the procedures in...

A common-environment OpenVMS Cluster that consists of newly installed systems

VSI OpenVMS System Manager's Manual to build these files. Because the files on new operating systems are empty except for the Digital-supplied accounts, very little coordination is necessary.

An OpenVMS Cluster that will combine one or more computers that have been running with computer-specific files

Appendix B, Building Common Files to create common copies of the files from the computer-specific files.

5.8.1. Procedure

In a common-environment cluster with one common system disk, you use a common copy of each system file and place the files in the SYS$COMMON:[SYSEXE] directory on the common system disk or on a disk that is mounted by all cluster nodes. No further action is required.

To prepare a common user environment for an OpenVMS Cluster system that includes more than one common OpenVMS Integrity server system disk or more than one common OpenVMS Alpha system disk, you must coordinate the system files on those disks.

Rules: The following rules apply to the procedures described in Table 5.4, “Procedure for Coordinating Files”:
  • Disks holding common resources must be mounted early in the system startup procedure, such as in the SYLOGICALS.COM procedure.

  • You must ensure that the disks are mounted with each OpenVMS Cluster reboot.


Table 5.4. Procedure for Coordinating Files
StepAction

1

Decide where to locate the SYSUAF.DAT and NETPROXY.DAT files. In a cluster with multiple system disks, system management is much easier if the common system files are located on a single disk that is not a system disk.

2

Copy SYS$SYSTEM:SYSUAF.DAT and SYS$SYSTEM:NETPROXY.DAT to a location other than the system disk.

3

Copy SYS$SYSTEM:RIGHTSLIST.DAT and SYS$SYSTEM:VMSMAIL_PROFILE.DATA to the same directory in which SYSUAF.DAT and NETPROXY.DAT reside.

4

Edit the file SYS$COMMON:[SYSMGR]SYLOGICALS.COM on each system disk and define logical names that specify the location of the cluster common files.

Example: If the files will be located on $1$DGA16, define logical names as follows:
$ DEFINE/SYSTEM/EXEC SYSUAF -
      $1$DGA16:[VMS$COMMON.SYSEXE]SYSUAF.DAT
$ DEFINE/SYSTEM/EXEC NETPROXY -
      $1$DGA16:[VMS$COMMON.SYSEXE]NETPROXY.DAT
$ DEFINE/SYSTEM/EXEC RIGHTSLIST -
      $1$DGA16:[VMS$COMMON.SYSEXE]RIGHTSLIST.DAT
$ DEFINE/SYSTEM/EXEC VMSMAIL_PROFILE -
      $1$DGA16:[VMS$COMMON.SYSEXE]VMSMAIL_PROFILE.DATA
$ DEFINE/SYSTEM/EXEC NETNODE_REMOTE -
      $1$DGA16:[VMS$COMMON.SYSEXE]NETNODE_REMOTE.DAT
$ DEFINE/SYSTEM/EXEC NETNODE_UPDATE -
      $1$DGA16:[VMS$COMMON.SYSMGR]NETNODE_UPDATE.COM
$ DEFINE/SYSTEM/EXEC QMAN$MASTER -
      $1$DGA16:[VMS$COMMON.SYSEXE]
5
To ensure that the system disks are mounted correctly with each reboot, follow these steps:
  1. Copy the SYS$EXAMPLES:CLU_MOUNT_DISK.COM file to the [VMS$COMMON.SYSMGR] directory, and edit it for your configuration.

  2. Edit SYLOGICALS.COM and include commands to mount, with the appropriate volume label, the system disk containing the shared files.

    Example: If the system disk is $1$DGA16, include the following command:


$ @SYS$SYSDEVICE:[VMS$COMMON.SYSMGR]CLU_MOUNT_DISK.COM $1$DGA16: volume-label
6

When you are ready to start the queuing system, be sure you have moved the queue and journal files to a cluster-available disk. Any cluster common disk is a good choice if the disk has sufficient space.

Enter the following command:
$ START/QUEUE/MANAGER $1$DGA16:[VMS$COMMON.SYSEXE]

5.8.2. Network Database Files

In OpenVMS Cluster systems on the LAN and in mixed-interconnect clusters, you must also coordinate the SYS$MANAGER:NETNODE_UPDATE.COM file, which is a file that contains all essential network configuration data for satellites. NETNODE_UPDATE.COM is updated each time you add or remove a satellite or change its Ethernet or FDDI hardware address. This file is discussed more thoroughly in Section 10.4.2, “Satellite Network Data”.

In OpenVMS Cluster systems configured with DECnet for OpenVMS software, you must also coordinate NETNODE_REMOTE.DAT, which is the remote node network database.

5.9. System Time on the Cluster

When a computer joins the cluster, the cluster attempts to set the joining computer's system time to the current time on the cluster. Although it is likely that the system time will be similar on each cluster computer, there is no assurance that the time will be set. Also, no attempt is made to ensure that the system times remain similar throughout the cluster. (For example, there is no protection against different computers having different clock rates).

An OpenVMS Cluster system spanning multiple time zones must use a single, clusterwide common time on all nodes. Use of a common time ensures timestamp consistency (for example, between applications, file-system instances) across the OpenVMS Cluster members.

5.9.1. Setting System Time

Use the SYSMAN command CONFIGURATION SET TIME to set the time across the cluster. This command issues warnings if the time on all nodes cannot be set within certain limits. Refer to the VSI OpenVMS System Manager's Manual for information about the SET TIME command.

Chapter 6. Cluster Storage Devices

One of the most important features of OpenVMS Cluster systems is the ability to provide access to devices and files across multiple systems.

In a traditional computing environment, a single system is directly attached to its storage subsystems. Even though the system may be networked with other systems, when the system is shut down, no other system on the network has access to its disks or any other devices attached to the system.

In an OpenVMS Cluster system, disks and tapes can be made accessible to one or more members. So, if one computer shuts down, the remaining computers still have access to the devices.

6.1. Data File Sharing

Cluster-accessible devices play a key role in OpenVMS Clusters because, when you place data files or applications on a cluster-accessible device, computer scan share a single copy of each common file. Data sharing is possible between Integrity server systems, between:
  • Integrity servers

  • Integrity servers and AlphaServer systems

  • AlphaServer systems

In addition, multiple systems that are permitted in the same OpenVMS Cluster system can write to a shared disk file simultaneously. It is this ability that allows multiple systems in an OpenVMS Cluster to share a single system disk; multiple systems can boot from the same system disk and share operating system files and utilities to save disk space and simplify system management.

Note

Tapes do not allow multiple systems to access a tape file simultaneously.

6.1.1. Access Methods

Depending on your business needs, you may want to restrict access to a particular device to the users on the computer that are directly connected (local) to the device. Alternatively, you may decide to set up a disk or tape as a served device so that any user on any OpenVMS Cluster computer can allocate and use it.

Table 6.1, “Device Access Methods” describes the various access methods.
Table 6.1. Device Access Methods
MethodDevice AccessCommentsIllustrated in

Local

Restricted to the computer that is directly connected to the device.

Can be set up to be served to other systems.

Figure 6.3, “Configuration with Cluster-Accessible Devices”

Dual ported

Using either of two physical ports, each of which can be connected to separate controllers. A dual-ported disk can survive the failure of a single controller by failing over to the other controller.

As long as one of the controllers is available, the device is accessible by all systems in the cluster.

Figure 6.1, “Dual-Ported Disks”

Shared

Through a shared interconnect to multiple systems.

Can be set up to be served to systems that are not on the shared interconnect.

Figure 6.2, “Dual-Pathed Disks”

Served

Through a computer that has the MSCP or TMSCP server software loaded.

MSCP and TMSCP serving are discussed in Section 6.3, “MSCP and TMSCP Served Disks and Tapes”.

Figures 6.2 and 6.3

Dual pathed

Possible through more than one path.

If one path fails, the device is accessed over the other path. Requires the use of allocation classes (described in Section 6.2.1, “Allocation Classes”to provide a unique, path-independent name.)

Figure 6.2, “Dual-Pathed Disks”

Note: The path to an individual disk may appear to be local from some nodes and served from others.

6.1.2. Examples

When storage subsystems are connected directly to a specific system, the availability of the subsystem is lower due to the reliance on the host system. To increase the availability of these configurations, OpenVMS Cluster systems support dual porting, dual pathing, and MSCP and TMSCP serving.

Figure 6.1, “Dual-Ported Disks” shows a dual-ported configuration, in which the disks have independent connections to two separate computers. As long as one of the computers is available, the disk is accessible by the other systems in the cluster.

Figure 6.1. Dual-Ported Disks
Dual-Ported Disks

Note

Disks can be shadowed using Volume Shadowing for OpenVMS. The automatic recovery from system failure provided by dual porting and shadowing is transparent to users and does not require any operator intervention.

Figure 6.2, “Dual-Pathed Disks” shows a dual-pathed FC and Ethernet configuration. The disk devices, accessible through a shared SCSI interconnect, are MSCP served to the client nodes on the LAN.

Rule: A dual-pathed DSA disk cannot be used as a system disk for a directly connected CPU. Because a device can be on line to one controller at a time, only one of the server nodes can use its local connection to the device. The second server node accesses the device through the MSCP (or the TMSCP server). If the computer that is currently serving the device fails, the other computer detects the failure and fails the device over to its local connection. The device thereby remains available to the cluster.

Figure 6.2. Dual-Pathed Disks
Dual-Pathed Disks
Dual-pathed disks or tapes can be failed over between two computers that serve the devices to the cluster, provided that:
  • The same device controller letter is generated and the same allocation class is specified on each computer, with the result that the device has the same name on both systems. (Section 6.2.1, “Allocation Classes” describes allocation classes).

  • Both computers are running the MSCP server for disks, the TMSCP server for tapes, or both.

Caution

Failure to observe these requirements can endanger data integrity.

You can set up HSG or HSV storage devices to be dual ported between two storage subsystems, as shown in Figure 6.3, “Configuration with Cluster-Accessible Devices”.

Figure 6.3. Configuration with Cluster-Accessible Devices
Configuration with Cluster-Accessible Devices

By design, HSG and HSV disks and tapes are directly accessible by all OpenVMS Cluster nodes that are connected to the same star coupler. Therefore, if the devices are dual ported, they are automatically dual pathed. Computers connected by FC can access a dual-ported HSG or HSV device by way of a path through either subsystem connected to the device. If one subsystem fails, access fails over to the other subsystem.

Note

To control the path that is taken during failover, you can specify a preferred path to force access to disks over a specific path. Section 6.1.3, “Specifying a Preferred Path” describes the preferred-path capability.

See the Guidelines for OpenVMS Cluster Configurations, for more information on FC storage devices.

6.1.3. Specifying a Preferred Path

The operating system supports specifying a preferred path for DSA disks, including RA series disks and disks that are accessed through the MSCP server. (This function is not available for tapes.) If a preferred path is specified for a disk, the MSCP disk class drivers use that path:
  • For the first attempt to locate the disk and bring it on line with a DCL command MOUNT

  • For failover of an already mounted disk

In addition, you can initiate failover of a mounted disk to force the disk to the preferred path or to use load-balancing information for disks accessed by MSCP servers.

You can specify the preferred path by using the SET PREFERRED_PATH DCL command or by using the $QIO function (IO$_SETPRFPATH),with the P1 parameter containing the address of a counted ASCII string(.ASCIC). This string is the node name of the HSG or HSV, or of the OpenVMS system that is to be the preferred path.

Rule: The node name must match an existing node running the MSCP server that is known to the local node.

Reference: For more information about the use of the SET PREFERRED_PATH DCL command, refer to the VSI OpenVMS DCL Dictionary.

For more information about the use of the IO$_SETPRFPATH function, refer to the VSI OpenVMS I/O User's Reference Manual.

6.2. Naming OpenVMS Cluster Storage Devices

Note

The naming convention of Fibre Channel devices is documented in the Guidelines for OpenVMS Cluster Configurations. The naming of all other devices is described in this section.

In the OpenVMS operating system, a device name takes the form of ddcu, where:
  • dd represents the predefined code for the device type

  • c represents the predefined controller designation

  • u represents the unit number

For SCSI, the controller letter is assigned by OpenVMS, based on the system configuration. The unit number is determined by the SCSI bus ID and the logical unit number (LUN) of the device.

Because device names must be unique in an OpenVMS Cluster, and because every cluster member must use the same name for the same device, OpenVMS adds a prefix to the device name, as follows:
  • If a device is attached to a single computer, the device name is extended to include the name of that computer:
    node$ddcu

    where node represents the SCS node name of the system on which the device resides.

  • If a device is attached to multiple computers, the node name part of the device name is replaced by a dollar sign and a number (called a node or port allocation class, depending on usage),as follows:
    $allocation-class$ddcu
  • SAS disks follow the device naming similar to that of SCSI devices, that is, Target-LUN numbering. So a disk on SAS target ID 1 and LUN 0 will be named as DKA100. For SAS tapes you can use the Fibre channel naming convention, that is, DGAtxx: The SYSGEN parameter SAS_NAMING can be used to use SCSI numbering in tapes also.

6.2.1. Allocation Classes

The purpose of allocation classes is to provide unique and unchanging device names. The device name is used by the OpenVMS Cluster distributed lock manager in conjunction with OpenVMS facilities (such as RMS and the XQP) to uniquely identify shared devices, files, and data.

Allocation classes are required in OpenVMS Cluster configurations where storage devices are accessible through multiple paths. Without the use of allocation classes, device names that relied on node names would change as access paths to the devices change.

Prior to OpenVMS Version 7.1, only one type of allocation class existed, which was node based. It was named allocation class. OpenVMS Version 7.1 introduced a second type, port allocation class, which is specific to a single interconnect and is assigned to all devices attached to that interconnect. Port allocation classes were originally designed for naming SCSI devices. Their use has been expanded to include additional devices types: floppy disks, PCI RAID controller disks, and IDE disks.

The use of port allocation classes is optional. They are designed to solve the device-naming and configuration conflicts that can occur in certain configurations, as described in Section 6.2.3, “Reasons for Using Port Allocation Classes”.

To differentiate between the earlier node-based allocation class and the newer port allocation class, the term node allocation classwas assigned to the earlier type.

Prior to OpenVMS Version 7.2, all nodes with direct access to the same multipathed device were required to use the same nonzero value for the node allocation class. OpenVMS Version 7.2 introduced the MSCP_SERVE_ALL system parameter, which can be set to serve all disks or to exclude those whose node allocation class differs.

Note

If SCSI devices are connected to multiple hosts and if port allocation classes are not used, then all nodes with direct access to the same multipathed devices must use the same nonzero node allocation class.

Multipathed MSCP controllers also have an allocation class parameter, which is set to match that of the connected nodes. (If the allocation class does not match, the devices attached to the nodes cannot be served).

6.2.2. Specifying Node Allocation Classes

A node allocation class can be assigned to computers, HSG or HSV controllers. The node allocation class is a numeric value from 1 to 255 that is assigned by the system manager.

The default node allocation class value is 0. A node allocation class value of 0 is appropriate only when serving a local, single-pathed disk. If a node allocation class of 0 is assigned, served devices are named using the node-name$device-name syntax, that is, the device name prefix reverts to the node name.

The following rules apply to specifying node allocation class values:
  1. When serving satellites, the same nonzero node allocation class value must be assigned to the serving computers and controllers.

  2. All cluster-accessible devices on computers with a nonzero node allocation class value must have unique names throughout the cluster. For example, if two computers have the same node allocation class value, it is invalid for both computers to have a local disk named DGA0 or a tape named MUA0. This also applies to HSG and HSV subsystems.

System managers provide node allocation classes separately for disks and tapes. The node allocation class for disks and the node allocation class for tapes can be different.

The node allocation class names are constructed as follows:
$disk-allocation-class$device-name
$tape-allocation-class$device-name

Caution: Failure to set node allocation class values and device unit numbers correctly can endanger data integrity and cause locking conflicts that suspend normal cluster operations.

Figure 6.5, “Device Names in a Mixed-Interconnect Cluster” includes satellite nodes that access devices $1$DUA17 and $1$MUA12 through the JUPITR and NEPTUN computers. In this configuration, the computers JUPITR and NEPTUN require node allocation classes so that the satellite nodes are able to use consistent device names regardless of the access path to the devices.

Note: System management is usually simplified by using the same node allocation class value for all servers, HSG and HSV subsystems; you can arbitrarily choose a number between 1 and 255. Note, however, that to change a node allocation class value, you must shutdown and reboot the entire cluster (described in Section 8.6, “Post-configuration Tasks”). If you use a common node allocation class for computers and controllers, ensure that all devices have unique unit numbers.

6.2.2.1. Assigning Node Allocation Class Values on Computers

There are two ways to assign a node allocation class: by using CLUSTER_CONFIG.COM or CLUSTER_CONFIG_LAN.COM, which is described in Section 8.4, “Changing Computer Characteristics”, or by using AUTOGEN, as shown in the following table.
StepAction

1

Edit the root directory [SYS n.SYSEXE]MODPARAMS.DAT on each node that boots from the system disk. The following example shows a MODPARAMS.DAT file. The entries are hypothetical and should be regarded as examples, not as suggestions for specific parameter settings.
!
! Site-specific AUTOGEN data file. In an OpenVMS Cluster
! where a common system disk is being used, this file
! should reside in SYS$SPECIFIC:[SYSEXE], not a common
! system directory.
!
! Add modifications that you want to make to AUTOGEN’s
! hardware configuration data, system parameter
! calculations, and page, swap, and dump file sizes
! to the bottom of this file.
SCSNODE="NODE01"
SCSSYSTEMID=99999
NISCS_LOAD_PEA0=1
VAXCLUSTER=2
MSCP_LOAD=1
MSCP_SERVE_ALL=1
ALLOCLASS=1
TAPE_ALLOCLASS=1

2

Invoke AUTOGEN to set the system parameter values:
$ @SYS$UPDATE:AUTOGEN start-phase end-phase

3

Shut down and reboot the entire cluster in order for the new values to take effect.

6.2.2.2. Node Allocation Class Example With a DSA Disk and Tape

Figure 6.4, “Disk and Tape Dual Pathed Between Computers ” shows a DSA disk and tape that are dual pathed between two computers.

Figure 6.4. Disk and Tape Dual Pathed Between Computers
Disk and Tape Dual Pathed Between Computers
In this configuration:
  • URANUS and NEPTUN access the disk either locally or through the other computer's MSCP server.

  • When satellites ARIEL and OBERON access $1$DGA8, a path is made through either URANUS or NEPTUN.

  • If, for example, the node URANUS has been shut down, the satellites can access the devices through NEPTUN. When URANUS reboots, access is available through either URANUS or NEPTUN.

6.2.2.3. Node Allocation Class Example With Mixed Interconnects

Figure 6.5, “Device Names in a Mixed-Interconnect Cluster” shows how device names are typically specified in a mixed-interconnect cluster. This figure also shows how relevant system parameter values are set for each FC computer.

Figure 6.5. Device Names in a Mixed-Interconnect Cluster
Device Names in a Mixed-Interconnect Cluster
In this configuration:
  • A disk and a tape are dual pathed to the HSG or HSV subsystems named VOYGR1 and VOYGR2; these subsystems are connected to JUPITR, SATURN, URANUS and NEPTUN through the star coupler.

  • The MSCP and TMSCP servers are loaded on JUPITR and NEPTUN (MSCP_LOAD = 1, TMSCP_LOAD = 1) and the ALLOCLASS and TAPE_ALLOCLASS parameters are set to the same value (1) on these computers and on both HSG or HSV subsystems.

Note: For optimal availability, two or more FC connected computers can serve HSG or HSV devices to the cluster.

6.2.2.4. Node Allocation Classes and RAID Array 210 and 230 Devices

If you have RAID devices connected to StorageWorks RAID Array 210 or 230 subsystems, you might experience device-naming problems when running in a cluster environment if nonzero node allocation classes are used. In this case, the RAID devices will be named $ n$DRcu, where n is the (nonzero) node allocation class, c is the controller letter, and u is the unit number.

If multiple nodes in the cluster have the same (nonzero) node allocation class and these same nodes have RAID controllers, then RAID devices that are distinct might be given the same name (for example, $1$DRA0). This problem can lead to data corruption.

To prevent such problems, use the DR_UNIT_BASE system parameter, which causes the DR devices to be numbered sequentially, starting with the DR_UNIT_BASE value that you specify. For example, if the node allocation class is $1, the controller letter is A, and you set DR_UNIT_BASE on one cluster member to 10, the first device name generated by the RAID controller will be $1$DRA10, followed by $1$DRA11, $1$DRA12, and so forth.

To ensure unique DR device names, set the DR_UNIT_BASE number on each cluster member so that the resulting device numbers do not overlap. For example, you can set DR_UNIT_BASE on three cluster members to 10, 20, and 30respectively. As long as each cluster member has 10 or fewer devices, the DR device numbers will be unique.

6.2.3. Reasons for Using Port Allocation Classes

When the node allocation class is nonzero, it becomes the device name prefix for all attached devices, whether the devices are on a shared interconnect or not. To ensure unique names within a cluster, it is necessary for the ddcu part of the disk device name (for example, DKB0) to be unique within an allocation class, even if the device is on a private bus.

This constraint is relatively easy to overcome for DIGITAL Storage Architecture(DSA) devices, because a system manager can select from a large unit number space to ensure uniqueness. The constraint is more difficult to manage for other device types, such as SCSI devices whose controller letter and unit number are determined by the hardware configuration.

For example, in the configuration shown in Figure 6.6, “SCSI Device Names Using a Node Allocation Class”, each system has a private SCSI bus with adapter letter A. To obtain unique names, the unit numbers must be different. This constrains the configuration to a maximum of 8 devices on the two buses (or 16 if wide addressing can be used on one or more of the buses). This can result in empty StorageWorks drive bays and in a reduction of the system's maximum storage capacity.

Figure 6.6. SCSI Device Names Using a Node Allocation Class
SCSI Device Names Using a Node Allocation Class

6.2.3.1. Constraint of the SCSI Controller Letter in Device Names

The SCSI device name is determined in part by the SCSI controller through which the device is accessed (for example, B in DKB n). Therefore, to ensure that each node uses the same name for each device, all SCSI controllers attached to a shared SCSI bus must have the same OpenVMS device name. In Figure 6.6, “SCSI Device Names Using a Node Allocation Class”, each host is attached to the shared SCSI bus by controller PKB.

This requirement can make configuring a shared SCSI bus difficult, because a system manager has little or no control over the assignment of SCSI controller device names. It is particularly difficult to match controller letters on different system types when one or more of the systems have:
  • Built-in SCSI controllers that are not supported in SCSI clusters

  • Long internal cables that make some controllers inappropriate for SCSI clusters

6.2.3.2. Constraints Removed by Port Allocation Classes

The port allocation class feature has two major benefits:
  • A system manager can specify an allocation class value that is specific to a port rather than nodewide.

  • When a port has a nonzero port allocation class, the controller letter in the device name that is accessed through that port is always the letter A.

Using port allocation classes for naming SCSI, IDE, floppy disk, and PCI RAID controller devices removes the configuration constraints described in Section 6.2.2.4, “Node Allocation Classes and RAID Array 210 and 230 Devices”, in Section 6.2.3, “Reasons for Using Port Allocation Classes”, and in Section 6.2.3.1, “Constraint of the SCSI Controller Letter in Device Names”. You do not need to use the DR_UNIT_BASE system parameter recommended in Section 6.2.2.4, “Node Allocation Classes and RAID Array 210 and 230 Devices”. Furthermore, each bus can be given its own unique allocation class value, so the ddcu part of the disk device name (for example, DKB0) does not need to be unique across buses. Moreover, controllers with different device names can be attached to the same bus, because the disk device names no longer depend on the controller letter.

Figure 6.7, “Device Names Using Port Allocation Classes” shows the same configuration as Figure 6.6, “SCSI Device Names Using a Node Allocation Class”, with two additions: a host named CHUCK and an additional disk attached to the lower left SCSI bus. Portal location classes are used in the device names in this figure. A port allocation class of 116 is used for the SCSI interconnect that is shared, and port allocation class 0 is used for the SCSI interconnects that are not shared. By using port allocation classes in this configuration, you can do what was not allowed previously:
  • Attach an adapter with a name (PKA) that differs from the name of the other adapters (PKB) attached to the shared SCSI interconnect, as long as that port has the same port allocation class (116 in this example).

  • Use two disks with the same controller name and number (DKA300) be cause each disk is attached to a SCSI interconnect that is not shared.

Figure 6.7. Device Names Using Port Allocation Classes
Device Names Using Port Allocation Classes

6.2.4. Specifying Port Allocation Classes

A port allocation class is a designation for all ports attached to a single interconnect. It replaces the node allocation class in the device name.

The three types of port allocation classes are:
  • Port allocation classes of 1 to 32767 for devices attached to a multihost interconnect or a single-host interconnect, if desired

  • Port allocation class 0 for devices attached to a single-host interconnect

  • Port allocation class -1 when no port allocation class is in effect

Each type has its own naming rules.

6.2.4.1. Port Allocation Classes for Devices Attached to a Multi-Host Interconnect

The following rules pertain to port allocation classes for devices attached to a multihost interconnect:
  1. The valid range of port allocation classes is 1 through 32767.

  2. When using port allocation classes, the controller letter in the device name is always A, regardless of the actual controller letter. The $GETDVI item code DVI$_DISPLAY_DEVNAM displays the actual port name.

    Note that it is now more important to use fully specified names (for example, $101$DKA100 or ABLE$DKA100) rather than abbreviated names (such as DK100), because a system can have multiple DKA100 disks.

  3. Each port allocation class must be unique within a cluster.

  4. A port allocation class cannot duplicate the value of another node's tape or disk node allocation class.

  5. Each node for which MSCP serves a device should have the same nonzero allocation class value.

Examples of device names that use this type of port allocation class are shown in Table 6.2, “Examples of Device Names with Port Allocation Classes 1-32767 ”.
Table 6.2. Examples of Device Names with Port Allocation Classes 1-32767

Device Name

Description

$101$DKA0

The port allocation class is 101; DK represents the disk device category, A is the controller name, and 0 is the unit number.

$147$DKA0

The port allocation class is 147; DK represents the disk device category, A is the controller name, and 0 is the unit number.

6.2.4.2. Port Allocation Class 0 for Devices Attached to a Single-Host Interconnect

The following rules pertain to port allocation class 0 for devices attached to a single-host interconnect:
  1. Port allocation class 0 does not become part of the device name. Instead, the name of the node to which the device is attached becomes the first part of the device name.

  2. The controller letter in the device name remains the designation of the controller to which the device is attached. (It is not changed to A as it is for port allocation classes greater than zero.)

Examples of device names that use port allocation class 0 are shown in Table 6.3, “Examples of Device Names With Port Allocation Class 0”.
Table 6.3. Examples of Device Names With Port Allocation Class 0

Device Name

Description

ABLE$DKD100

ABLE is the name of the node to which the device is attached. D is the designation of the controller to which it is attached, not A as it is for port allocation classes with a nonzero class. The unit number of this device is 100. The port allocation class of $0$ is not included in the device name.

BAKER$DKC200

BAKER is the name of the node to which the device is attached, C is the designation of the controller to which it is attached, and 200 is the unit number. The port allocation class of $0$ is not included in the device name.

6.2.4.3. Port Allocation Class -1

The designation of port allocation class -1 means that a port allocation class is not being used. Instead, a node allocation class is used. The controller letter remains its predefined designation. (It is assigned by OpenVMS, based on the system configuration. It is not affected by a node allocation class).

6.2.4.4. How to Implement Port Allocation Classes

Port allocation classes were introduced in OpenVMS Alpha Version 7.1 with support in OpenVMS VAX. VAX computers can serve disks connected to Alpha systems that use port allocation classes in their names.

To implement port allocation classes, you must do the following:
  • Enable the use of port allocation classes.

  • Assign one or more port allocation classes.

  • At a minimum, reboot the nodes on the shared SCSI bus.

Enabling the Use of Port Allocation Classes

To enable the use of port allocation classes, you must set a new SYSGEN parameter DEVICE_NAMING to 1. The default setting for this parameter is zero. In addition, the SCSSYSTEMIDH system parameter must be set to zero. Check to make sure that it is.

Assigning Port Allocation Classes

You can assign one or more port allocation classes with the OpenVMS Cluster configuration procedure, CLUSTER_CONFIG.COM (or CLUSTER_CONFIG_LAN.COM).

If it is not possible to use CLUSTER_CONFIG.COM or CLUSTER_CONFIG_LAN.COM to assign port allocation classes (for example, if you are booting a private system disk into an existing cluster), you can use the new SYSBOOT SET/CLASS command.

The following example shows how to use the new SYSBOOT SET/CLASS command to assign an existing port allocation class of 152 to port PKB.
SYSBOOT> SET/CLASS PKB 152

The SYSINIT process ensures that this new name is used in successive boots.

To deassign a port allocation class, enter the port name without a class number. For example:
SYSBOOT> SET/CLASS PKB

The mapping of ports to allocation classes is stored in SYS$SYSTEM:SYS$DEVICES.DAT, a standard text file. You use the CLUSTER_CONFIG.COM (or CLUSTER_CONFIG_LAN.COM) command procedure or, in special cases, SYSBOOT to change SYS$DEVICES.DAT.

6.2.4.5. Clusterwide Reboot Requirements for SCSI Interconnects

Changing a device's allocation class changes the device name. A clusterwide reboot ensures that all nodes see the device under its new name, which in turn means that the normal device and file locks remain consistent.

Rebooting an entire cluster when a device name changes is not mandatory. You may be able to reboot only the nodes that share the SCSI bus, as described in the following steps. The conditions under which you can do this and the results that follow are also described.
  1. Dismount the devices whose names have changed from all nodes.

    This is not always possible. In particular, you cannot dismount a disk on nodes where it is the system disk. If the disk is not dismounted, a subsequent attempt to mount the same disk using the new device name will fail with the following error:
    %MOUNT-F-VOLALRMNT, another volume of same label already mounted

    Therefore, you must reboot any node that cannot dismount the disk.

  2. Reboot all nodes connected to the SCSI bus.

    Before you reboot any of these nodes, make sure the disks on the SCSI bus are dismounted on the nodes not rebooting.

    Note

    OpenVMS ensures that a node cannot boot if the result is a SCSI bus with naming different from another node already accessing the same bus. (This check is independent of the dismount check in step 1).

    After the nodes that are connected to the SCSI bus reboot, the device exists with its new name.

  3. Mount the devices systemwide or clusterwide.

    If no other node has the disk mounted under the old name, you can mount the disk systemwide or clusterwide using its new name. The new device name will be seen on all nodes running compatible software, and these nodes can also mount the disk and access it normally.

    Nodes that have not rebooted still see the old device name as well as the new device name. However, the old device name cannot be used; the device, when accessed by the old name, is off line. The old name persists until the node reboots.

6.3. MSCP and TMSCP Served Disks and Tapes

The MSCP server and the TMSCP server make locally connected disks and tapes available to all cluster members. Locally connected disks and tapes are not automatically cluster accessible. Access to these devices is restricted to the local computer unless you explicitly set them up as cluster accessible using the MSCP server for disks or the TMSCP server for tapes.

6.3.1. Enabling Servers

To make a disk or tape accessible to all OpenVMS Cluster computers, the MSCP or TMSCP server must be:

Table 6.4. MSCP_LOAD and TMSCP_LOAD Parameter Settings

Parameter

Value

Meaning

MSCP_LOAD

0

Do not load the MSCP_SERVER. This is the default.

1

Load the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter using the default CPU load capacity.

>1

Load the MSCP server with attributes specified by the MSCP_SERVE_ALL parameter. Use the MSCP_LOAD value as the CPU load capacity.

TMSCP_LOAD

0

Do not load the TMSCP server and do not serve any tapes (default value).

1

Load the TMSCP server and serve all available tapes, including all local tapes and all multihost tapes with a matching TAPE_ALLOCLASS value.

Table 6.5, “MSCP_SERVE_ALL and TMSCP_SERVE_ALL Parameter Settings” summarizes the system parameter values you can specify for MSCP_SERVE_ALL and TMSCP_SERVE_ALL to configure the MSCP and TMSCP servers. Initial values are determined by your responses when you execute the installation or upgrade procedure or when you execute the CLUSTER_CONFIG.COM command procedure described in Chapter 8, Configuring an OpenVMS Cluster System to set up your configuration.

Starting with OpenVMS Version 7.2, the serving types are implemented as a bit mask. To specify the type of serving your system will perform, locate the type you want in Table 6.5, “MSCP_SERVE_ALL and TMSCP_SERVE_ALL Parameter Settings” and specify its value. For some systems, you may want to specify two serving types, such as serving the system disk and serving locally attached disks. To specify such a combination, add the values of each type, and specify the sum.

Note

In a mixed-version cluster that includes any systems running OpenVMS Version 7.1- x or earlier, serving all available disks is restricted to serving all disks whose allocation class matches the system's node allocation class (pre-Version 7.2 meaning).To specify this type of serving, use the value 9 (which sets bit 0 and bit 3).


Table 6.5. MSCP_SERVE_ALL and TMSCP_SERVE_ALL Parameter Settings

Parameter

Bit

Value When Set

Meaning

MSCP_SERVE_ALL

0

1

Serve all available disks (locally attached and those connected to HS x and DSSI controllers). Disks with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are also served if bit 3 is not set.

1

2

Serve locally attached (non-HS x and non-DSSI)disks. The server does not monitor its I/O traffic and does not participate in load balancing.

2

4

Serve the system disk. This is the default setting. This setting is important when other nodes in the cluster rely on this system being able to serve its system disk. This setting prevents obscure contention problems that can occur when a system attempts to complete I/O to a remote system disk whose system has failed. For more information, see Section 6.3.1.1, “Serving the System Disk”.

3

8

Restrict the serving specified by bit 0. All disks except those with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are served.

This is pre-Version 7.2 behavior. If your cluster includes systems running Open 7.1- x or earlier, and you want to serve all available disks, you must specify 9, the result of setting this bit and bit 0.

4

15

By default, the bit 4 is not set, hence the DUDRIVER will accept the devices with unit number greater than 9999. On the client side, if bit 4 is set (10000 binary) in the MSCP_SERVE_ALL parameter, the client will reject devices with unit number greater than 9999 and retains the earlier behavior.

TMSCP_SERVE_ALL

0

1

Serve all available tapes (locally attached and those connected to HS x and DSSI controllers). Tapes with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter) are also served if bit 3 is not set.

1

2

Serve locally attached (non-HS x and non-DSSI) tapes.

3

8

Restrict the serving specified by bit 0. Serve all tapes except those with allocation classes that differ from the system's allocation class (set by the ALLOCLASS parameter).

This is pre-Version 7.2 behavior. If your cluster includes systems running OpenVMS Version 7.1- x or earlier, and you want to serve all available tapes, you must specify 9, the result of setting this bit and bit 0.

4

15

By default, the bit 4 is not set, hence the TUDRIVER will accept the devices with unit number greater than 9999. On the client side, if bit 4 is set (10000 binary) in the TMSCP_SERVE_ALL parameter, the client will reject devices with unit number greater than 9999 and retains the earlier behavior.

Although the serving types are now implemented as a bit mask, the values of 0, 1, and 2, specified by bit 0 and bit 1, retain their original meanings. These values are shown in the following table:

Value

Description

0

Do not serve any disks (tapes). This is the default.

1

Serve all available disks (tapes).

2

Serve only locally attached (non-HS x and non-DSSI) disks (tapes).

6.3.1.1. Serving the System Disk

Setting bit 2 of the MSCP_SERVE_ALL system parameter to serve the system disk is important when other nodes in the cluster rely on this system being able to serve its system disk. This setting prevents obscure contention problems that can occur when a system attempts to complete I/O to are mote system disk whose system has failed.

The following sequence of events describes how a contention problem can occur if serving the system disk is disabled (that is, if bit 2 is not set):
  • The MSCP_SERVE_ALL setting is changed to disable serving when the system reboots.

  • The serving system crashes.

  • The client system that was executing I/O to the serving system's system disk is holding locks on resources of that system disk.

  • The client system starts mount verification.

  • The serving system attempts to boot but cannot because of the locks held on its system disk by the client system.

  • The client's mount verification process times out after a period of time set by the MVTIMEOUT system parameter, and the client system releases the locks. The time period could be several hours.

  • The serving system is able to reboot.

6.3.1.2. Setting the MSCP and TMSCP System Parameters

Use either of the following methods to set these system parameters:
  • Specify appropriate values for these parameters in a computer's MODPARAMS.DAT file and then run AUTOGEN.

  • Run the CLUSTER_CONFIG.COM or the CLUSTER_CONFIG_LAN.COM procedure, as appropriate, and choose the CHANGE option to perform these operations for disks and tapes.

With either method, the served devices become accessible when the serving computer reboots. Further, the servers automatically serve any suitable device that is added to the system later. For example, if new drives are attached to an HSC subsystem, the devices are dynamically configured.

Note: The SCSI retention command modifier is not supported by the TMSCP server. Retention operations should be performed from the node serving the tape.

6.4. MSCP I/O Load Balancing

MSCP I/O load balancing offers the following advantages:
  • Faster I/O response

  • Balanced work load among the members of an OpenVMS Cluster

Two types of MSCP I/O load balancing are provided by OpenVMS Cluster software: static and dynamic. Static load balancing occurs on Integrity servers and Alpha systems and are based on the load capacity ratings of the server systems.

6.4.1. Load Capacity

The load capacity ratings for Integrity servers and Alpha systems are predetermined by VSI. These ratings are used in the calculation of the available serving capacity for MSCP static and dynamic load balancing. You can override these default settings by specifying a different load capacity with the MSCP_LOAD parameter.

Note that the MSCP server load-capacity values (either the default value or the value you specify with MSCP_LOAD) are estimates used by the load-balancing feature. They cannot change the actual MSCP serving capacity of a system.

A system's MSCP serving capacity depends on many factors including its power, the performance of its LAN adapter, and the impact of other processing loads. The available serving capacity, which is calculated by each MSCP server as described in Section 6.4.2, “Available Serving Capacity”, is used solely to bias the selection process when a client system (for example, a satellite) chooses which server system to use when accessing a served disk.

6.4.2. Available Serving Capacity

The load-capacity ratings are used by each MSCP server to calculate its available serving capacity.

The available serving capacity is calculated in the following way:

Step

Calculation

1

Each MSCP server counts the read and write requests sent to it and periodically converts this value to requests per second.

2

Each MSCP server subtracts its requests per second from its load capacity to compute its available serving capacity.

6.4.3. Static Load Balancing

MSCP servers periodically send their available serving capacities to the MSCP class driver(DUDRIVER). When a disk is mounted or one fails over, DUDRIVER assigns the server with the highest available serving capacity to it. (TMSCP servers do not perform this monitoring function.) This initial assignment is called static load balancing.

6.4.4. Overriding MSCP I/O Load Balancing for Special Purposes

In some configurations, you may want to designate one or more systems in your cluster as the primary I/O servers and restrict I/O traffic on other systems. You can accomplish these goals by overriding the default load-capacity ratings used by the MSCP server. For example, if your cluster consists of two Alpha systems and one VAX 6000-400 system and you want to reduce the MSCP served I/O traffic to the VAX, you can assign the VAX a low MSCP_LOAD value, such as 50. Because the two Alpha systems each start with a load-capacity rating of 340 and the VAX now starts with a load-capacity rating of 50, the MSCP served satellites will direct most of the I/O traffic to the Alpha systems.

6.5. Managing Cluster Disks With the Mount Utility

For locally connected disks to be accessible to other nodes in the cluster, the MSCP server software must be loaded on the computer to which the disks are connected (see Section 6.3.1, “Enabling Servers”). Further, each disk must be mounted with the Mount utility, using the appropriate qualifier: /CLUSTER, /SYSTEM, or /GROUP. Mounting multiple disks can be automated with command procedures; a sample command procedure, MSCPMOUNT.COM, is provided in the SYS$EXAMPLES directory on your system.

The Mount utility also provides other qualifiers that determine whether a disk is automatically rebuilt during a remount operation. Different rebuilding techniques are recommended for data and system disks.

This section describes how to use the Mount utility for these purposes.

6.5.1. Mounting Cluster Disks

To mount disks that are to be shared among all computers, specify the MOUNT command as shown in the following table.

IF...

THEN...

At system startup

The disk is attached to a single system and is to be made available to all other nodes in the cluster.

Use MOUNT/CLUSTER device-name on the computer to which the disk is to be mounted. The disk is mounted on every computer that is active in the cluster at the time the command executes. First, the disk is mounted locally. Then, if the mount operation succeeds, the disk is mounted on other nodes in the cluster.

The computer has no disks directly attached to it.

Use MOUNT/SYSTEM device-name on the computer for each disk the computer needs to access. The disks can be attached to a single system or shared disks that are accessed by an HS xcontroller. Then, if the mount operation succeeds, the disk is mounted on the computer joining the cluster.

When the system is running

You want to add a disk.

Use MOUNT/CLUSTER device-name on the computer to which the disk is to be mounted. The disk is mounted on every computer that is active in the cluster at the time the command executes. First, the disk is mounted locally. Then, if the mount operation succeeds, the disk is mounted on other nodes in the cluster.

To ensure disks are mounted whenever possible, regardless of the sequence that systems in the cluster boot (or shut down), startup command procedures should use MOUNT/CLUSTER and MOUNT/SYSTEM as described in the preceding table.

Note: Only system or group disks can be mounted across the cluster or on a subset of the cluster members. If you specify MOUNT/CLUSTER without the /SYSTEM or /GROUP qualifier, /SYSTEM is assumed. Also note that each cluster disk mounted with the /SYSTEM or /GROUP qualifier must have a unique volume label.

6.5.2. Examples of Mounting Shared Disks

Suppose you want all the computers in a three-member cluster to share a disk named COMPANYDOCS. To share the disk, one of the three computers can mount COMPANYDOCS using the MOUNT/CLUSTER command, as follows:
$ MOUNT/CLUSTER/NOASSIST $1$DUA4: COMPANYDOCS
If you want just two of the three computers to share the disk, those two computers must both mount the disk with the same MOUNT command, as follows:
$ MOUNT/SYSTEM/NOASSIST $1$DUA4: COMPANYDOCS

To mount the disk at startup time, include the MOUNT command either in a common command procedure that is invoked at startup time or in the computer-specific startup command file.

Note: The /NOASSIST qualifier is used in command procedures that are designed to make several attempts to mount disks. The disks may be temporarily offline or otherwise not available for mounting. If, after several attempts, the disk cannot be mounted, the procedure continues. The /ASSIST qualifier, which is the default, causes a command procedure to stop and query the operator if a disk cannot be mounted immediately.

6.5.3. Mounting Cluster Disks With Command Procedures

To configure cluster disks, you can create command procedures to mount them. You may want to include commands that mount cluster disks in a separate command procedure file that is invoked by a site-specific SYSTARTUP procedure. Depending on your cluster environment, you can set up your command procedure in either of the following ways:
  • As a separate file specific to each computer in the cluster by making copies of the common procedure and storing them as separate files

  • As a common computer-independent file on a shared disk

With either method, each computer can invoke the common procedure from the site-specific SYSTARTUP procedure.

Example: The MSCPMOUNT.COM file in the SYS$EXAMPLES directory on your system is a sample command procedure that contains commands typically used to mount cluster disks. The example includes comments explaining each phase of the procedure.

6.5.4. Disk Rebuild Operation

To minimize disk I/O operations (and thus improve performance) when files are created or extended, the OpenVMS file system maintains a cache of preallocated file headers and disk blocks.

If a disk is dismounted improperly—for example, if a system fails or is removed from a cluster without running SYS$SYSTEM:SHUTDOWN.COM—this preallocated space becomes temporarily unavailable. When the disk is remounted, MOUNT scans the disk to recover the space. This is called a disk rebuild operation.

6.5.5. Rebuilding Cluster Disks

On a nonclustered computer, the MOUNT scan operation for recovering preallocated space merely prolongs the boot process. In an OpenVMS Cluster system, however, this operation can degrade response time for all user processes in the cluster. While the scan is in progress on a particular disk, most activity on that disk is blocked.

Note: User processes that attempt to read or write to files on the disk can experience delays of several minutes or longer, especially if the disk contains a large number of files or has many users.

Because the rebuild operation can delay access to disks during the startup of any OpenVMS Cluster computer, VSI recommends that procedures for mounting cluster disks use the /NOREBUILD qualifier. When MOUNT/NOREBUILD is specified, disks are not scanned to recover lost space, and users experience minimal delays while computers are mounting disks.

Reference: Section 6.5.6, “Rebuilding System Disks” provides information about rebuilding system disks. Section 9.7.1, “Avoiding Disk Rebuilds ” provides more information about disk rebuilds and system-disk throughput techniques.

6.5.6. Rebuilding System Disks

Rebuilding system disks is especially critical because most system activity requires access to a system disk. When a system disk rebuild is in progress, very little activity is possible on any computer that uses that disk.

Unlike other disks, the system disk is automatically mounted early in the boot sequence. If a rebuild is necessary, and if the value of the system parameter ACP_REBLDSYSD is 1, the system disk is rebuilt during the boot sequence. (The default setting of 1 for the ACP_REBLDSYSD system parameter specifies that the system disk should be rebuilt.) Exceptions are as follows:

Setting

Comments

ACP_REBLDSYSD parameter should be set to 0 on satellites.

This setting prevents satellites from rebuilding a system disk when it is mounted early in the boot sequence and eliminates delays caused by such a rebuild when satellites join the cluster.

ACP_REBLDSYSD should be set to the default value of 1 on boot servers, and procedures that mount disks on the boot servers should use the /REBUILD qualifier.

While these measures can make boot server rebooting more noticeable, they ensure that system disk space is available after an unexpected shutdown.

Once the cluster is up and running, system managers can submit a batch procedure that executes SET VOLUME/REBUILD commands to recover lost disk space. Such procedures can run at a time when users would not be inconvenienced by the blocked access to disks (for example, between midnight and 6 a.m. each day). Because the SET VOLUME/REBUILD command determines whether a rebuild is needed, the procedures can execute the command for each disk that is usually mounted.

Suggestion: The procedures run more quickly and cause less delay in disk access if they are executed on:
  • Powerful computers

  • Computers that have direct access to the volume to be rebuilt

Moreover, several such procedures, each of which rebuilds a different set of disks, can be executed simultaneously.

Caution: If either or both of the following conditions are true when mounting disks, it is essential to run a procedure with SET VOLUME/REBUILD commands on a regular basis to rebuild the disks:
  • Disks are mounted with the MOUNT/NOREBUILD command.

  • The ACP_REBLDSYSD system parameter is set to 0.

Failure to rebuild disk volumes can result in a loss of free space and in subsequent failures of applications to create or extend files.

6.6. Shadowing Disks Across an OpenVMS Cluster

Volume shadowing (sometimes referred to as disk mirroring) achieves high data availability by duplicating data on multiple disks. If one disk fails, the remaining disk or disks can continue to service application and user I/O requests.

6.6.1. Purpose

Volume Shadowing for OpenVMS software provides data availability across the full range of OpenVMS configurations—from single nodes to large OpenVMS Cluster systems—so you can provide data availability where you need it most.

Volume Shadowing for OpenVMS software is an implementation of RAID 1 (redundant arrays of independent disks) technology. Volume Shadowing for OpenVMS prevents a disk device failure from interrupting system and application operations. By duplicating data on multiple disks, volume shadowing transparently prevents your storage subsystems from becoming a single point of failure because of media deterioration, communication path failure, or controller or device failure.

6.6.2. Shadow Sets

You can mount up to six compatible disk volumes to form a shadow set. Figure 6.8, “Shadow Set With Three Members” shows three compatible disk volumes used to form a shadow set. Each disk in the shadow set is known as a shadow set member. Volume Shadowing for OpenVMS logically binds the shadow set devices together and represents them as a single virtual device called a virtual unit. This means that the multiple members of the shadow set, represented by the virtual unit, appear to operating systems and users as a single, highly available disk.

Figure 6.8. Shadow Set With Three Members
Shadow Set With Three Members

6.6.3. I/O Capabilities

Applications and users read and write data to and from a shadow set using the same commands and program language syntax and semantics that are used for nonshadowed I/O operations. System managers manage and monitor shadow sets using the same commands and utilities they use for nonshadowed disks. The only difference is that access is through the virtual unit, not to individual devices.

Reference: VSI OpenVMS Volume Shadowing Guide describes the shadowing product capabilities in detail.

6.6.4. Supported Devices

For a single workstation or a large data centre, valid shadowing configurations include:
  • All MSCP compliant DSA drives

  • All SAS devices

  • All StorageWorks SCSI disks and controllers, and some third-party SCSI devices that implement READL (read long) and WRITEL (write long) commands and use the SCSI disk driver (DKDRIVER)

    Restriction: SCSI disks that do not support READL and WRITEL are restricted because these disks do not support the shadowing data repair (disk bad-block errors) capability. Thus, using unsupported SCSI disks can cause members to be removed from the shadow set.

You can shadow data disks and system disks. Thus, a system disk need not be a single point of failure for any system that boots from that disk. System disk shadowing becomes especially important for OpenVMS Cluster systems that use a common system disk from which multiple computers boot.

Volume Shadowing for OpenVMS does not support the shadowing of quorum disks. This is because volume shadowing makes use of the OpenVMS distributed lock manager, and the quorum disk must be utilized before locking is enabled.

There are no restrictions on the location of shadow set members beyond the valid disk configurations defined in the Volume Shadowing for OpenVMS Software Product Description.

6.6.5. Shadow Set Limits

You can mount a default maximum of 500 shadow sets (each having one to six members) in a standalone system or OpenVMS Cluster system. If more than 500 shadow sets are required, the SYSGEN parameter SHADOW_MAX_UNIT must be increased. The number of shadow sets supported is independent of controller and device types. The shadow sets can be mounted as public or private volumes.

For any changes to these limits, consult the Volume Shadowing for OpenVMS Software Product Description.

6.6.6. Distributing Shadowed Disks

The controller-independent design of shadowing allows you to manage shadow sets regardless of their controller connection or location in the OpenVMS Cluster system and helps provide improved data availability and very flexible configurations.

For clusterwide shadowing, members can be located anywhere in an OpenVMS Cluster system and served by MSCP servers across any supported OpenVMS Cluster interconnect.

Figure 6.9, “Shadow Sets Accessed Through the MSCP Server” shows how shadow set member units are on line to local controllers located on different nodes. In the figure, a disk volume is local to each of the nodes ATABOY and ATAGRL. The MSCP server provides access to the shadow set members over the LAN or IP network. Even though the disk volumes are local to different nodes, the disks are members of the same shadow set. A member unit that is local to one node can be accessed by the remote node over the MSCP server.

Figure 6.9. Shadow Sets Accessed Through the MSCP Server
Shadow Sets Accessed Through the MSCP Server

For shadow sets that are mounted on an OpenVMS Cluster system, mounting or dismounting a shadow set on one node in the cluster does not affect applications or user functions executing on other nodes in the system. For example, you can dismount the virtual unit from one node in an OpenVMS Cluster system and leave the shadow set operational on the remaining nodes on which it is mounted.

Other shadowing notes:
  • If an individual disk volume is already mounted as a member of an active shadow set, the disk volume cannot be mounted as a standalone disk on another node at the same time.

  • System disks can be shadowed. All nodes booting from shadowed system disks must:
    • Have a Volume Shadowing for OpenVMS license.

    • Set shadowing system parameters to enable shadowing and specify the system disk virtual unit number.

Chapter 7. Setting Up and Managing Cluster Queues

This chapter discusses queuing topics specific to OpenVMS Cluster systems. Because queues in an OpenVMS Cluster system are established and controlled with the same commands used to manage queues on a standalone computer, the discussions in this chapter assume some knowledge of queue management on a standalone system, as described in the VSI OpenVMS System Manager's Manual.

Note

See the VSI OpenVMS System Manager's Manual for information about queuing compatibility.

7.1. Introduction

Users can submit jobs to any queue in the OpenVMS Cluster system, regardless of the processor on which the job will actually execute. Generic queues can balance the work load among the available processors.

The system manager can use one or several queue managers to manage batch and print queues for an entire OpenVMS Cluster system. Although a single queue manager is sufficient for most systems, multiple queue managers can be useful for distributing the batch and print work load across nodes in the cluster.

Once the batch and print queue characteristics are set up, the system manager can rely on the distributed queue manager to make queues available across the cluster.

7.2. Controlling Queue Availability

The distributed queue manager prevents the queuing system from being affected when a node enters or leaves the cluster during cluster transitions. The following table describes how the distributed queue manager works.

WHEN...

THEN...

Comments

The node on which the queue manager is running leaves the OpenVMS Cluster system.

The queue manager automatically fails over to another node.

This failover occurs transparently to users on the system.

Nodes are added to the cluster.

The queue manager automatically serves the new nodes.

The system manager does not need to enter a command explicitly to start queuing on the new node.

The OpenVMS Cluster system reboots.

The queuing system automatically restarts by default.

Thus, you do not have to include commands in your startup command procedure for queuing.

The operating system automatically restores the queuing system with the parameters defined in the queuing database.

This is because when you start the queuing system, the characteristics you define are stored in a queue database.

To control queues, the queue manager maintains a clusterwide queue database that stores information about queues and jobs. Whether you use one or several queue managers, only one queue database is shared across the cluster. Keeping the information for all processes in one database allows jobs submitted from any computer to execute on any queue (provided that the necessary mass storage devices are accessible).

7.3. Starting a Queue Manager and Creating the Queue Database

You start up a queue manager using the START/QUEUE/MANAGER command as you would on a standalone computer. However, in an OpenVMS Cluster system, you can also provide a failover list and a unique name for the queue manager. The /NEW_VERSION qualifier creates a new queue database.

The following command example shows how to start a queue manager:
$ START/QUEUE/MANAGER/NEW_VERSION/ON=(GEM,STONE,*)
The following table explains the components of this sample command.

Command

Function

START/QUEUE/MANAGER

Creates a single, clusterwide queue manager named SYS$QUEUE_MANAGER.

/NEW_VERSION

Creates a new queue database in SYS$COMMON:[SYSEXE] that consists of the following three files:
  • QMAN$MASTER.DAT (master file)

  • SYS$QUEUE_MANAGER.QMAN$QUEUES (queue file)

  • SYS$QUEUE_MANAGER.QMAN$JOURNAL (journal file)

Rule: Use the /NEW_VERSION qualifier only on the first invocation of the queue manager or if you want to create a new queue database.

/ON= (node-list) [optional]

Specifies an ordered list of nodes that can claim the queue manager if the node running the queue manager should exit the cluster. In the example:
  • The queue manager process starts on node GEM.

  • If the queue manager is running on node GEM and GEM leaves the cluster, the queue manager fails over to node STONE.

  • The asterisk wildcard (*) is specified as the last node in the node list to indicate that any remaining, unlisted nodes can start the queue manager in any order.

    Rules: Complete node names are required; you cannot specify the asterisk wildcard character as part of a node name.

    If you want to exclude certain nodes from being eligible to run the queue manager, do not use the asterisk wildcard character in the node list.

/NAME_OF_MANAGER [optional]

Allows you to assign a unique name to the queue manager. Unique queue manager names are necessary if you run multiple queue managers. For example, using the /NAME_OF_MANAGER qualifier causes queue and journal files to be created using the queue manager name instead of the default name SYS$QUEUE_MANAGER. For example, adding the /NAME_OF_MANAGER=PRINT_MANAGER qualifier command creates these files:
  • QMAN$MASTER.DAT
  • PRINT_MANAGER.QMAN$QUEUES
  • PRINT_MANAGER.QMAN$JOURNAL
Rules for OpenVMS Cluster systems with multiple system disks:
  • Specify the locations of both the master file and the queue and journal files for systems that do not boot from the system disk where the files are located.

    Reference: If you want to locate the queue database files on other devices or directories, refer to the VSI OpenVMS System Manager's Manual for instructions.

  • Specify a device and directory that is accessible across the OpenVMS Cluster.

  • Define the device and directory identically in the SYS$COMMON:SYLOGICALS.COM startup command procedure on every node.

7.4. Starting Additional Queue Managers

Running multiple queue managers balances the work load by distributing batch and print jobs across the cluster. For example, you might create separate queue managers for batch and print queues in clusters with CPU or memory shortages. This allows the batch queue manager to run on one node while the print queue manager runs on a different node.

7.4.1. Command Format

To start additional queue managers, include the /ADD and /NAME_OF_MANAGER qualifiers on the START/QUEUE/MANAGER command. Do not specify the /NEW_VERSION qualifier. For example:
$ START/QUEUE/MANAGER/ADD/NAME_OF_MANAGER=BATCH_MANAGER

7.4.2. Database Files

Multiple queue managers share one QMAN$MASTER.DAT master file, but an additional queue file and journal file is created for each queue manager. The additional files are named in the following format, respectively:
  • name_of_manager.QMAN$QUEUES

  • name_of_manager.QMAN$JOURNAL

By default, the queue database and its files are located in SYS$COMMON:[SYSEXE]. If you want to relocate the queue database files, refer to the instructions in Section 7.6, “Moving Queue Database Files”.

7.5. Stopping the Queuing System

When you enter the STOP/QUEUE/MANAGER/CLUSTER command, the queue manager remains stopped, and requests for queuing are denied until you enter the START/QUEUE/MANAGER command (without the /NEW_VERSION qualifier).

The following command shows how to stop a queue manager named PRINT_MANAGER:
$ STOP/QUEUE/MANAGER/CLUSTER/NAME_OF_MANAGER=PRINT_MANAGER

Rule: You must include the /CLUSTER qualifier on the command line whether or not the queue manager is running on an OpenVMS Cluster system. If you omit the/CLUSTER qualifier, the command stops all queues on the default node without stopping the queue manager. (This has the same effect as entering the STOP/QUEUE/ON_NODE command).

7.6. Moving Queue Database Files

The files in the queue database can be relocated from the default location of SYS$COMMON:[SYSEXE] to any disk that is mounted clusterwide or that is accessible to the computers participating in the clusterwide queue scheme. For example, you can enhance system performance by locating the database on a shared disk that has a low level of activity.

7.6.1. Location Guidelines

The master file QMAN$MASTER can be in a location separate from the queue and journal files, but the queue and journal files must be kept together in the same directory. The queue and journal files for one queue manager can be separate from those of other queue managers.

The directory you specify must be available to all nodes in the cluster. If the directory specification is a concealed logical name, it must be defined identically in the SYS$COMMON:SYLOGICALS.COM startup command procedure on every node in the cluster.

Reference: The VSI OpenVMS System Manager's Manual contains complete information about creating or relocating the queue database files. See also Section 7.12, “Using a Common Command Procedure” for a sample common procedure that sets up an OpenVMS Cluster batch and print system.

7.7. Setting Up Print Queues

To establish print queues, you must determine the type of queue configuration that best suits your OpenVMS Cluster system. You have several alternatives that depend on the number and type of print devices you have on each computer and on how you want print jobs to be processed. For example, you need to decide:
  • Which print queues you want to establish on each computer

  • Whether to set up any clusterwide generic queues to distribute print job processing across the cluster

  • Whether to set up auto start queues for availability or improved startup time

Once you determine the appropriate strategy for your cluster, you can create your queues. Figure 7.1, “Sample Printer Configuration” shows the printer configuration for a cluster consisting of the active computers JUPITR, SATURN, and URANUS.

Figure 7.1. Sample Printer Configuration
Sample Printer Configuration

7.7.1. Creating a Queue

You set up OpenVMS Cluster print queues using the same method that you would use for a standalone computer. However, in an OpenVMS Cluster system, you must provide a unique name for each queue you create.

7.7.2. Command Format

You create and name a print queue by specifying the INITIALIZE/QUEUE command at the DCL prompt in the following format:
$ INITIALIZE/QUEUE/ON=node-name::device[/START][/NAME_OF_MANAGER=
name-of-manager] queue-name

Qualifier

Description

/ON

Specifies the computer and printer to which the queue is assigned. If you specify the /START qualifier, the queue is started.

/NAME_OF_MANAGER

If you are running multiple queue managers, you should also specify the queue manager with the qualifier.

7.7.3. Ensuring Queue Availability

You can also use the auto start feature to simplify startup and ensure high availability of execution queues in an OpenVMS Cluster. If the node on which the autostart queue is running leaves the OpenVMS Cluster, the queue automatically fails over to the next available node on which autostart is enabled. Autostart is particularly useful on LAT queues. Because LAT printers are usually shared among users of multiple systems or in OpenVMS Cluster systems, many users are affected if a LAT queue is unavailable.

Format for creating autostart queues:

Create an autostart queue with a list of nodes on which the queue can run by specifying the DCL command INITIALIZE/QUEUE in the following format:
INITIALIZE/QUEUE/AUTOSTART_ON=(node-name::device:,node-name::device:, . . . queue-name

When you use the /AUTOSTART_ON qualifier, you must initially activate the queue for autostart, either by specifying the /START qualifier with the INITIALIZE/QUEUE command or by entering a START/QUEUE command. However, the queue cannot begin processing jobs until the ENABLE AUTOSTART/QUEUES command is entered for a node on which the queue can run. Generic queues cannot be autostart queues.

Rules: Generic queues cannot be autostart queues. Note that you cannot specify both /ON and /AUTOSTART_ON.

Reference: Refer to Section 7.13, “Disabling Autostart During Shutdown” for information about setting the time at which autostart is disabled.

7.7.4. Examples

The following commands make the local print queue assignments for JUPITR shown in Figure 7.2, “Print Queue Configuration” and start the queues:
$ INITIALIZE/QUEUE/ON=JUPITR::LPA0/START/NAME_OF_MANAGER=PRINT_MANAGER JUPITR_LPA0
$ INITIALIZE/QUEUE/ON=JUPITR::LPB0/START/NAME_OF_MANAGER=PRINT_MANAGER JUPITR_LPB0
Figure 7.2. Print Queue Configuration
Print Queue Configuration

7.8. Setting Up Clusterwide Generic Print Queues

The clusterwide queue database enables you to establish generic queues that function throughout the cluster. Jobs queued to clusterwide generic queues are placed in any assigned print queue that is available, regardless of its location in the cluster. However, the file queued for printing must be accessible to the computer to which the printer is connected.

7.8.1. Sample Configuration

Figure 7.3, “Clusterwide Generic Print Queue Configuration” illustrates a clusterwide generic print queue in which the queues for all LPA0 printers in the cluster are assigned to a clusterwide generic queue named SYS$PRINT.

A clusterwide generic print queue needs to be initialized and started only once. The most efficient way to start your queues is to create a common command procedure that is executed by each OpenVMS Cluster computer (see Section 7.12.3, “Example”).

Figure 7.3. Clusterwide Generic Print Queue Configuration
Clusterwide Generic Print Queue Configuration

7.8.2. Command Example

The following command initializes and starts the clusterwide generic queue SYS$PRINT:
$ INITIALIZE/QUEUE/GENERIC=(JUPITR_LPA0,SATURN_LPA0,URANUS_LPA0)/START SYS$PRINT

Jobs queued to SYS$PRINT are placed in whichever assigned print queue is available. Thus, in this example, a print job from JUPITR that is queued to SYS$PRINT can be queued to JUPITR_LPA0, SATURN_LPA0, or URANUS_LPA0.

7.9. Setting Up Execution Batch Queues

Generally, you set up execution batch queues on each OpenVMS Cluster computer using the same procedures you use for a standalone computer. For more detailed information about how to do this, see the VSI OpenVMS System Manager's Manual.

7.9.1. Before You Begin

Before you establish batch queues, you should decide which type of queue configuration best suits your cluster. As system manager, you are responsible for setting up batch queues to maintain efficient batch job processing on the cluster. For example, you should do the following:
  • Determine what type of processing will be performed on each computer.

  • Set up local batch queues that conform to these processing needs.

  • Decide whether to set up any clusterwide generic queues that will distribute batch job processing across the cluster.

  • Decide whether to use autostart queues for startup simplicity.

Once you determine the strategy that best suits your needs, you can create a command procedure to set up your queues. Figure 7.4, “Sample Batch Queue Configuration” shows a batch queue configuration for a cluster consisting of computers JUPITR, SATURN, and URANUS.

Figure 7.4. Sample Batch Queue Configuration
Sample Batch Queue Configuration

7.9.2. Batch Command Format

You create a batch queue with a unique name by specifying the DCL command INITIALIZE/QUEUE/BATCH in the following format
$ INITIALIZE/QUEUE/BATCH/ON=node::[/START][/NAME_OF_MANAGER=
name-of-manager] queue-name

Qualifier

Description

/ON

Specifies the computer on which the batch queue runs.

/START

Starts the queue.

/NAME_OF_MANAGER

Specifies the name of the queue manager if you are running multiple queue managers.

7.9.3. Autostart Command Format

You can initialize and start an autostart batch queue by specifying the DCL command INITIALIZE/QUEUE/BATCH. Use the following command format:
INITIALIZE/QUEUE/BATCH/AUTOSTART_ON=node::queue-name

When you use the /AUTOSTART_ON qualifier, you must initially activate the queue for autostart, either by specifying the /START qualifier with the INITIALIZE/QUEUE command or by entering a START/QUEUE command. However, the queue cannot begin processing jobs until the ENABLE AUTOSTART/QUEUES command is entered for a node on which the queue can run.

Rule: Generic queues cannot be autostart queues. Note that you cannot specify both /ON and /AUTOSTART_ON.

7.9.4. Examples

The following commands make the local batch queue assignments for JUPITR, SATURN, and URANUS shown in Figure 7.4, “Sample Batch Queue Configuration”:
$ INITIALIZE/QUEUE/BATCH/ON=JUPITR::/START/NAME_OF_MANAGER=BATCH_QUEUE JUPITR_BATCH
$ INITIALIZE/QUEUE/BATCH/ON=SATURN::/START/NAME_OF_MANAGER=BATCH_QUEUE SATURN_BATCH
$ INITIALIZE/QUEUE/BATCH/ON=URANUS::/START/NAME_OF_MANAGER=BATCH_QUEUE URANUS_BATCH

Because batch jobs on each OpenVMS Cluster computer are queued to SYS$BATCH by default, you should consider defining a logical name to establish this queue as a clusterwide generic batch queue that distributes batch job processing throughout the cluster (see Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”). Note, however, that you should do this only if you have a common-environment cluster.

7.10. Setting Up Clusterwide Generic Batch Queues

In an OpenVMS Cluster system, you can distribute batch processing among computers to balance the use of processing resources. You can achieve this workload distribution by assigning local batch queues to one or more clusterwide generic batch queues. These generic batch queues control batch processing across the cluster by placing batch jobs in assigned batch queues that are available. You can create a clusterwide generic batch queue as shown in Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”.

A clusterwide generic batch queue needs to be initialized and started only once. The most efficient way to perform these operations is to create a common command procedure that is executed by each OpenVMS Cluster computer (see Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”).

7.10.1. Sample Configuration

In Figure 7.5, “Clusterwide Generic Batch Queue Configuration”, batch queues from each OpenVMS Cluster computer are assigned to a clusterwide generic batch queue named SYS$BATCH. Users can submit a job to a specific queue (for example, JUPITR_BATCH or SATURN_BATCH), or, if they have no special preference, they can submit it by default to the clusterwide generic queue SYS$BATCH. The generic queue in turn places the job in an available assigned queue in the cluster.

If more than one assigned queue is available, the operating system selects the queue that minimizes the ratio (executing jobs/job limit) for all assigned queues.

Figure 7.5. Clusterwide Generic Batch Queue Configuration
Clusterwide Generic Batch Queue Configuration

7.11. Starting Local Batch Queues

Normally, you use local batch execution queues during startup to run batch jobs to start layered products. For this reason, these queues must be started before the ENABLE AUTOSTART command is executed, as shown in the command procedure in Example 7.1, “Sample Commands for Creating OpenVMS Cluster Queues”.

7.11.1. Startup Command Procedure

Start the local batch execution queue in each node's startup command procedure SYSTARTUP_VMS.COM. If you use a common startup command procedure, add commands similar to the following to your procedure:
$ SUBMIT/PRIORITY=255/NOIDENT/NOLOG/QUEUE=node_BATCH LAYERED_PRODUCT.COM
$ START/QUEUE node_BATCH
$ DEFINE/SYSTEM/EXECUTIVE SYS$BATCH node_BATCH

Submitting the startup command procedure LAYERED_PRODUCT.COM as a high-priority batch job before the queue starts ensures that the job is executed immediately, regardless of the job limit on the queue. If the queue is started before the command procedure was submitted, the queue might reach its job limit by scheduling user batch jobs, and the startup job would have to wait.

7.12. Using a Common Command Procedure

Once you have created queues, you must start them to begin processing batch and print jobs. In addition, you must make sure the queues are started each time the system reboots, by enabling autostart for autostart queues or by entering START/QUEUE commands for nonautostart queues. To do so, create a command procedure containing the necessary commands.

7.12.1. Command Procedure

You can create a common command procedure named, for example, QSTARTUP.COM, and store it on a shared disk. With this method, each node can share the same copy of the common QSTARTUP.COM procedure. Each node invokes the common QSTARTUP.COM procedure from the common version of SYSTARTUP. You can also include the commands to start queues in the common SYSTARTUP file instead of in a separate QSTARTUP.COM file.

7.12.2. Examples

Example 7.1, “Sample Commands for Creating OpenVMS Cluster Queues” shows commands used to create OpenVMS Cluster queues.
Example 7.1. Sample Commands for Creating OpenVMS Cluster Queues
$ 
1
$  DEFINE/FORM LN_FORM 10 /WIDTH=80 /STOCK=DEFAULT /TRUNCATE
$  DEFINE/CHARACTERISTIC 2ND_FLOOR 2
.
.
.
2
$  INITIALIZE/QUEUE/AUTOSTART_ON=(JUPITR::LPA0:)/START JUPITR_PRINT
$  INITIALIZE/QUEUE/AUTOSTART_ON=(SATURN::LPA0:)/START SATURN_PRINT
$  INITIALIZE/QUEUE/AUTOSTART_ON=(URANUS::LPA0:)/START URANUS_PRINT

.
.
.
3
$  INITIALIZE/QUEUE/BATCH/START/ON=JUPITR:: JUPITR_BATCH
$  INITIALIZE/QUEUE/BATCH/START/ON=SATURN:: SATURN_BATCH
$  INITIALIZE/QUEUE/BATCH/START/ON=URANUS:: URANUS_BATCH

.
.
.
4
$  INITIALIZE/QUEUE/START - 
_$ /AUTOSTART_ON=(JUPITR::LTA1:,SATURN::LTA1,URANUS::LTA1) -
_$ /PROCESSOR=LATSYM /FORM_MOUNTED=LN_FORM -
_$ /RETAIN=ERROR /DEFAULT=(NOBURST,FLAG=ONE,NOTRAILER) -
_$ /RECORD_BLOCKING LN03$PRINT
$ 
$  INITIALIZE/QUEUE/START -
_$ /AUTOSTART_ON=(JUPITR::LTA2:,SATURN::LTA2,URANUS::LTA2) -
_$ /PROCESSOR=LATSYM /RETAIN=ERROR -
_$ /DEFAULT=(NOBURST,FLAG=ONE,NOTRAILER) /RECORD_BLOCKING -
_$ /CHARACTERISTIC=2ND_FLOOR LA210$PRINT
$ 
5
$  ENABLE AUTOSTART/QUEUES/ON=SATURN
$  ENABLE AUTOSTART/QUEUES/ON=JUPITR
$  ENABLE AUTOSTART/QUEUES/ON=URANUS6
$  INITIALIZE/QUEUE/START SYS$PRINT -
_$ /GENERIC=(JUPITR_PRINT,SATURN_PRINT,URANUS_PRINT)
$ 
7
$  INITIALIZE/QUEUE/BATCH/START SYS$BATCH -
_$ /GENERIC=(JUPITR_BATCH,SATURN_BATCH,URANUS_BATCH)
$ 
Following are descriptions of each command or group of commands in Example 7.1, “Sample Commands for Creating OpenVMS Cluster Queues”.

1

Define all printer forms and characteristics.

2

Initialize local print queues. In the example, these queues are autostart queues and are started automatically when the node executes the ENABLE AUTOSTART/QUEUES command. Although the /START qualifier is specified to activate the autostart queues, they do not begin processing jobs until autostart is enabled.

To enable autostart each time the system reboots, add the ENABLE AUTOSTART/QUEUES command to your queue startup command procedure, as shown in Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”.

3

Initialize and start local batch queues on all nodes, including satellite nodes. In this example, the local batch queues are not autostart queues.

4

Initialize queues for remote LAT printers. In the example, these queues are autostart queues and are set up to run on one of three nodes. The queues are started on the first of those three nodes to execute the ENABLE AUTOSTART command.

You must establish the logical devices LTA1 and LTA2 in the LAT startup command procedure LAT$SYSTARTUP.COM on each node on which the autostart queue can run. For more information, see the description of editing LAT$SYSTARTUP.COM in the VSI OpenVMS System Manager's Manual.

Although the /START qualifier is specified to activate these autostart queues, they will not begin processing jobs until autostart is enabled.


5

Enable autostart to start the autostart queues automatically. In the example, autostart is enabled on node SATURN first, so the queue manager starts the autostart queues that are set up to run on one of several nodes.


6

Initialize and start the generic output queue SYS$PRINT. This is a nonautostart queue (generic queues cannot be autostart queues). However, generic queues are not stopped automatically when a system is shut down, so you do not need to restart the queue each time a node reboots.


7

Initialize and start the generic batch queue SYS$BATCH. Because this is a generic queue, it is not stopped when the node shuts down. Therefore, you do not need to restart the queue each time a node reboots.

7.12.3. Example

Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues” illustrates the use of a common QSTARTUP command procedure on a shared disk.
Example 7.2. Common Procedure to Start OpenVMS Cluster Queues
$! 
$! QSTARTUP.COM -- Common procedure to set up cluster queues 
$! 
$! 
1
$ NODE = F$GETSYI("NODENAME") 
$! 
$! Determine the node-specific subroutine 
$! 
$ IF (NODE .NES. "JUPITR") .AND. (NODE .NES. "SATURN") .AND. (NODE .NES. - 
  "URANUS")
$    THEN  
$       GOSUB SATELLITE_STARTUP 
$    ELSE 
2
$! 
$!      Configure remote LAT devices. 
$! 
$       SET TERMINAL LTA1: /PERM /DEVICE=LN03 /WIDTH=255 /PAGE=60 - 
            /LOWERCASE /NOBROAD 
$       SET TERMINAL LTA2: /PERM /DEVICE=LA210 /WIDTH=255 /PAGE=66 - 
            /NOBROAD 
$       SET DEVICE LTA1: /SPOOLED=(LN03$PRINT,SYS$SYSDEVICE:) 
$       SET DEVICE LTA2: /SPOOLED=(LA210$PRINT,SYS$SYSDEVICE:) 
3
$       START/QUEUE/BATCH 'NODE'_BATCH 
$       GOSUB 'NODE'_STARTUP 
$    ENDIF 
$ GOTO ENDING 
$!              
$! Node-specific subroutines start here 
$! 
4
$ SATELLITE_STARTUP: 
$! 
$! Start a batch queue for satellites. 
$!                     
$ START/QUEUE/BATCH 'NODE'_BATCH 
$ RETURN 
$!           
5
$JUPITR_STARTUP: 
$! 
$! Node-specific startup for JUPITR:: 
$! Setup local devices and start nonautostart queues here 
$! 
$ SET PRINTER/PAGE=66 LPA0: 
$ RETURN 
$! 
$SATURN_STARTUP: 
$! 
$! Node-specific startup for SATURN:: 
$! Setup local devices and start nonautostart queues here 
$! 
   .
   .
   .
$ RETURN 
$! 
$URANUS_STARTUP: 
$! 
$! Node-specific startup for URANUS:: 
$! Setup local devices and start nonautostart queues here 
$! 
   .
   .
   .
$ RETURN 
$!     
$ENDING:  
6
$! Enable autostart to start all autostart queues 
$! 
$ ENABLE AUTOSTART/QUEUES 
$ EXIT
Following are descriptions of each phase of the common QSTARTUP.COM command procedure in Example 7.2, “Common Procedure to Start OpenVMS Cluster Queues”.

1

Determine the name of the node executing the procedure.


2

On all large nodes, set up remote devices connected by the LAT. The queues for these devices are autostart queues and are started automatically when the ENABLE AUTOSTART/QUEUES command is executed at the end of this procedure.

In the example, these autostart queues were set up to run on one of three nodes. The queues start when the first of those nodes executes the ENABLEAUTOSTART/QUEUES command. The queue remains running as long as one of those nodes is running and has autostart enabled.


3

On large nodes, start the local batch queue. In the example, the local batch queues are nonautostart queues and must be started explicitly with START/QUEUE commands.


4

On satellite nodes, start the local batch queue.

5

Each node executes its own subroutine. On node JUPITR, set up the line printer device LPA0:. The queue for this device is an autostart queue and is started automatically when the ENABLE AUTOSTART/QUEUES command is executed.


6

Enable autostart to start all autostart queues.

7.13. Disabling Autostart During Shutdown

By default, the shutdown procedure disables autostart at the beginning of the shutdown sequence. Autostart is disabled to allow autostart queues with failover lists to fail over to another node. Autostart also prevents any autostart queue running on another node in the cluster to fail over to the node being shutdown.

7.13.1. Options

You can change the time at which autostart is disabled in the shutdown sequence in one of two ways:

Option

Description

1

Define the logical name SHUTDOWN$DISABLE_AUTOSTART as follows:
$ DEFINE/SYSTEM/EXECUTIVE SHUTDOWN$DISABLE_AUTOSTART 
number-of-minutes

Set the value of number-of-minutes to the number of minutes before shutdown when autostart is to be disabled. You can add this logical name definition to SYLOGICALS.COM. The value of number-of-minutes is the default value for the node. If this number is greater than the number of minutes specified for the entire shutdown sequence, autostart is disabled at the beginning of the sequence.

2

Specify the DISABLE_AUTOSTART number-of-minutes option during the shutdown procedure. (The value you specify for number-of-minutes overrides the value specified for the SHUTDOWN$DISABLE_AUTOSTART logical name).

Reference: See the VSI OpenVMS System Manager's Manual for more information about changing the time at which autostart is disabled during the shutdown sequence.

Chapter 8. Configuring an OpenVMS Cluster System

This chapter provides an overview of the cluster configuration command procedures and describes the preconfiguration tasks required before running either command procedure. Then it describes each major function of the command procedures and the post-configuration tasks, including running AUTOGEN.COM.

8.1. Overview of the Cluster Configuration Procedures

Two similar command procedures are provided for configuring and reconfiguring an OpenVMS Cluster system: CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. The choice depends on whether you use the LANCP utility or DECnet for satellite booting in your cluster. CLUSTER_CONFIG_LAN.COM provides satellite booting services with the LANCP utility; CLUSTER_CONFIG.COM provides satellite booting services with DECnet.

Also, to configure an Integrity server system use CLUSTER_CONFIG_LAN.COM and to configure an Alpha system, use either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM. You can use only CLUSTER_CONFIG_LAN.COM for configuring Cluster over IP.

In a satellite environment, you may want to determine which command procedure is used for configuring a cluster. To determine whether CLUSTER_CONFIG or CLUSTER_CONFIG_LAN is used in cluster configuration, see the SYS$SYSTEM:MODPARAMS.DAT file. While configuring a cluster, the command procedure name is added as a comment in the MODPARAMS.DAT file.

During the ADD operation, a comment similar to the following is added to MODPARAMS.DAT for CLUSTER_CONFIG:
! CLUSTER_CONFIG creating for ADD operation on 4-APR-2009 14:21:00.89
For CLUSTER_CONFIG_LAN:
! CLUSTER_CONFIG_LAN creating for ADD operation on 5-APR-2009 14:21:00.89 

Similar comments are added for the 'CHANGE' operation. For multiple entries in MODPARAMS.DAT, the last entry reflects the latest procedure name that is used to configure the cluster. See Section 4.5, “Configuring and Starting a Satellite Booting Service” for the factors to consider when choosing a satellite booting service.

These configuration procedures automate most of the tasks required to configure an OpenVMS Cluster system. When you invoke CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM, the following configuration options are displayed:
  • Add a computer to the cluster

  • Remove a computer from the cluster

  • Change a computer's characteristics

  • Create a duplicate system disk

  • Make a directory structure for a new root on a system disk

  • Delete a root from a system disk

By selecting the appropriate option, you can configure the cluster easily and reliably without invoking any OpenVMS utilities directly. Table 8.1, “Summary of Cluster Configuration Functions ” summarizes the functions that the configuration procedures perform for each configuration option.

The phrase cluster configuration command procedure, when used in this chapter, refers to both CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM. The questions of the two configuration procedures are identical except where they pertain to LANCP and DECnet.

Note: For help on any question in these command procedures, type a question mark (?) at the question.
Table 8.1. Summary of Cluster Configuration Functions
OptionFunctions Performed

ADD

Enables a node as a cluster member:
  • Establishes the new computer's root directory on a cluster common system disk and generates the computer's system parameter files, (IA64VMSSYS.PAR for Integrity server systems or ALPHAVMSSYS.PAR for Alpha systems), and MODPARAMS.DAT in its SYS$SPECIFIC:[SYSEXE] directory.

  • Generates the new computer's page and swap files (PAGEFILE.SYS and SWAPFILE.SYS).

  • Sets up a cluster quorum disk (optional).

  • Sets disk allocation class values, or port allocation class values (Alpha only), or both, with the ALLOCLASS parameter for the new computer, if the computer is being added as a disk server. If the computer is being added as a tape server, sets a tape allocation class value with the TAPE_ALLOCLASS parameter.

    Note: ALLOCLASS must be set to a value greater than zero if you are configuring an Alpha computer on a shared SCSI bus and you are not using a portal location class.

  • Generates an initial (temporary) startup procedure for the new computer. This initial procedure:
    • Runs NETCONFIG.COM to configure the network.

    • Runs AUTOGEN to set appropriate system parameter values for the computer.

    • Reboots the computer with normal startup procedures.

  • If the new computer is a satellite node, the configuration procedure updates:
    • Network databases for the computer on which the configuration procedure is executed to add the new computer.

    • SYS$MANAGER:NETNODE_UPDATE.COM command procedure on the local computer (as described in Section 10.4.2, “Satellite Network Data”).

REMOVE

Disables a node as a cluster member:
  • Deletes another computer's root directory and its contents from the local computer's system disk. If the computer being removed is a satellite, the cluster configuration command procedure updates SYS$MANAGER:NETNODE_UPDATE.COM on the local computer.

  • Updates the permanent and volatile remote node network databases on the local computer.

  • Removes the quorum disk.

CHANGE

Displays the CHANGE menu and prompts for appropriate information to:
  • Enable or disable the local computer as a disk server

  • Enable or disable the local computer as a boot server

  • Enable or disable IP for cluster communications on the local computer

  • Enable or disable the Ethernet or FDDI LAN for cluster communications on the local computer

  • Enable or disable a quorum disk on the local computer

  • Change a satellite's Ethernet or FDDI hardware address

  • Enable or disable the local computer as a tape server

  • Change the local computer's ALLOCLASS or TAPE_ALLOCLASS value

  • Change the local computer's shared SCSI port allocation class value

  • Enable or disable MEMORY CHANNEL for node-to-node cluster communications on the local computer

CREATE

Duplicates the local computer's system disk and removes all system roots from the new disk.

MAKE

Creates a directory structure for a new root on a system disk.

DELETE

Deletes a root from a system disk.

8.1.1. Before Configuring the System

Before invoking either the CLUSTER_CONFIG_LAN.COM or the CLUSTER_CONFIG.COM procedure to configure an OpenVMS Cluster system, perform the tasks described in Table 8.2, “Preconfiguration Tasks”.
Table 8.2. Preconfiguration Tasks
TaskProcedure

Determine whether the computer uses DECdtm.

When you add a computer to or remove a computer from a cluster that uses DECdtm services, there are a number of tasks you need to do in order to ensure the integrity of your data.

Reference: See the chapter about DECdtm services in the VSI OpenVMS System Manager's Manual for step-by-step instructions on setting up DECdtm in an OpenVMS Cluster system.

If you are not sure whether your cluster uses DECdtm services, enter this command sequence:
$ SET PROCESS /PRIVILEGES=SYSPRV
$ RUN SYS$SYSTEM:LMCP
LMCP> SHOW LOG

If your cluster does not use DECdtm services, the SHOW LOG command will display a file not found error message. If your cluster uses DECdtm services, it displays a list of the files that DECdtm uses to store information about transactions.

Ensure the network software providing the satellite booting service is up and running and all computers are connected to the LAN.

For nodes that will use the LANCP utility for satellite booting, run the LANCP utility and enter the LANCP command LIST DEVICE/MOPDLL to display a list of LAN devices on the system:
$ RUN SYS$SYSTEM:LANCP
LANCP> LIST DEVICE/MOPDLL
For nodes running DECnet for OpenVMS, enter the DCL command SHOW NETWORK to determine whether the network is up and running:
$ SHOW NETWORK
Product:  DECNET        Node:  CHBOSE               Address(es):  25.169
Product:  TCP/IP        Node:  chbose.ind.hp.com    Address(es):  18.156.235.23

This example shows that the node CHBOSE is running DECnet for OpenVMS and node chbose.ind.hp.com is running TCP/IP. If DECnet has not been started, the message SHOW-I-NONET, Network Unavailable is displayed.

For nodes running DECnet–Plus, refer to VSI OpenVMS DECnet Network Management Utilities for information about determining whether the DECnet–Plus network is up and running.

Select MOP and disk servers.

Every OpenVMS Cluster configured with satellite nodes must include at least one Maintenance Operations Protocol (MOP) and disk server. When possible, select multiple computers as MOP and disk servers. Multiple servers give better availability, and they distribute the work load across more LAN adapters.

Follow these guidelines when selecting MOP and disk servers:
  • Ensure that MOP servers have direct access to the system disk.

  • Ensure that disk servers have direct access to the storage that they are serving.

  • Choose the most powerful computers in the cluster. Low-powered computers can become overloaded when serving many busy satellites or when many satellites boot simultaneously. Note, however, that two or more moderately powered servers may provide better performance than a single high-powered server.

  • If you have several computers of roughly comparable power, it is reasonable to use them all as boot servers. This arrangement gives optimal load balancing. In addition, if one computer fails or is shut down, others remain available to serve satellites.

  • After compute power, the most important factor in selecting a server is the speed of its LAN adapter. Servers should be equipped with the highest-bandwidth LAN adapters in the cluster.

  • If you are interconnecting the cluster using IP, note the local LAN adapter on which the IP address will be configured and used for SCS.

Make sure you are logged in to a privileged account.

Log in to a privileged account.

Rules: If you are adding a satellite, you must be logged into the system manager's account on a boot server. Note that the process privileges SYSPRV, OPER, CMKRNL, BYPASS, and NETMBX are required, because the procedure performs privileged system operations.

Coordinate cluster common files.

If your configuration has two or more system disks, follow the instructions in Chapter 5, Preparing a Shared Environment to coordinate the cluster common files.

Optionally, disable broadcast messages to your terminal.

While adding and removing computers, many such messages are generated. To disable the messages, you can enter the DCL command REPLY/DISABLE=(NETWORK, CLUSTER). See also Section 10.5, “Controlling OPCOM Messages” for more information about controlling OPCOM messages.

Predetermine answers to the questions asked by the cluster configuration procedure.

Table 8.3, “Data Requested by CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM” describes the data requested by the cluster configuration command procedures.

8.1.2. Data Requested by the Cluster Configuration Procedures

The following table describes the questions asked by the cluster configuration command procedures and describes how you might answer them. The table is supplied here so that you can determine answers to the questions before you invoke the procedure.

Because many of the questions are configuration specific, Table 8.3, “Data Requested by CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM” lists the questions according to configuration type, and not in the order they are asked.
Table 8.3. Data Requested by CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM
Information RequiredHow to Specify or Obtain

For all configurations

Device name of cluster system disk on which root directories will be created

Press Return to accept the default device name which is the translation of the SYS$SYSDEVICE: logical name, or specify a logical name that points to the common system disk.

Computer's root directory name on cluster system disk

Press Return to accept the procedure-supplied default, or specify a name in the form SYS x:
  • For both Integrity servers and Alpha systems with direct access to the system disk, the valid range of hexadecimal values is much larger. It includes both the VAX range of 1 through 9 or A through D, and also the range 10 through FFFF. Note that SYSE and SYSF are reserved for system use.

  • For satellites, x must be in the range of 10 through FFFF.

Workstation windowing system

System manager specifies. Workstation software must be installed before workstation satellites are added. If it is not, the procedure indicates that fact.

Location and sizes of page and swap files

This information is requested only when you add a computer to the cluster. Press Return to accept the default size and location. (The default sizes displayed in brackets by the procedure are minimum values. The default location is the device name of the cluster system disk).

If your configuration includes satellite nodes, you may realize a performance improvement by locating satellite page and swap files on a satellite's local disk, if such a disk is available. The potential for performance improvement depends on the configuration of your OpenVMS Cluster system disk and network.

To set up page and swap files on a satellite's local disk, the cluster configuration procedure creates a command procedure called SATELLITE_PAGE.COM in the satellite's [SYS n.SYSEXE] directory on the boot server's system disk. The SATELLITE_PAGE.COM procedure performs the following functions:
  • Mounts the satellite's local disk with a volume label that is unique in the cluster in the format node-name_SCSSYSTEMID.

    Reference: Refer to Section 8.6.5, “Altering Satellite Local Disk Labels” for information about altering the volume label.

  • Installs the page and swap files on the satellite's local disk.

Note: For page and swap disks that are shadowed, you must edit the MOUNT and INIT commands in SATELLITE_PAGE.COM to the appropriate syntax for mounting any specialized local disks (that is, host-based shadowing disks (DS xxx), or host-based RAID disks (DP xxxx), or DECram disks (MDA xxxx)) on the newly added node. CLUSTER_CONFIG(_LAN).COM does not create the MOUNT and INIT commands required for SHADOW, RAID, or DECram disks.

Note: To relocate the satellite's page and swap files (for example, from the satellite's local disk to the boot server's system disk, or the reverse) or to change file sizes:
  1. Create new PAGE and SWAP files on a shared device, as shown:
    $ MCR SYSGEN CREATE device:[dir] PAGEFILE.SYS/SIZE=
    block-count

    Note: If page and swap files will be created for a shadow set, you must edit SATELLITE_PAGE accordingly.

  2. Rename the SYS$SPECIFIC:[SYSEXE]PAGEFILE.SYS and SWAPFILE.SYS files to PAGEFILE.TMP and SWAPFILE.TMP.

  3. Reboot, and then delete the .TMP files.

  4. Modify the SYS$MANAGER:SYPAGSWPFILES.COM procedure to load the files.

Value for local computer's allocation class (ALLOCLASS or TAPE_ALLOCLASS) parameter.

The ALLOCLASS parameter can be used for a node allocation class or, on Alpha computers, a port allocation class. Refer to Section 6.2.1, “Allocation Classes” for complete information about specifying allocation classes.

Physical device name of quorum disk

System manager specifies.

For systems running DECnet for OpenVMS

Computer's DECnet node address for Phase IV

For the DECnet node address, you obtain this information as follows:

Computer's DECnet node name

Network manager supplies. The name must be from 1 to 6 alphanumeric characters and cannot include dollar signs ($) or underscores (_).

For systems running DECnet–Plus

Computer's DECnet node address for Phase IV(if you need Phase IV compatibility)

For the DECnet node address, you obtain this information as follows:

Node's DECnet full name

Determine the full name with the help of your network manager. Enter a string comprised of:
  • The namespace name, ending with a colon (:). This is optional.

  • The root directory, designated by a period (.).

  • Zero or more hierarchical directories, designated by a character string followed by a period (.).

  • The simple name, a character string that, combined with the directory names, uniquely identifies the node. For example:
    • .SALES.NETWORKS.MYNODE
    • MEGA:.INDIANA.JONES
    • COLUMBUS:.FLATWORLD

SCS node name for this node

Enter the OpenVMS Cluster node name, which is a string of 6or fewer alphanumeric characters.

DECnet synonym

Press Return to define a DECnet synonym, which is a short name for the node's full name. Otherwise, enter N.

Synonym name for this node

Enter a string of 6 or fewer alphanumeric characters. By default, it is the first 6 characters of the last simple name in the full name. For example:
  • Synonym: BLACKH

Note: The node synonym does not need to be the same as the OpenVMS Cluster node name.

MOP service client name for this node

Enter the name for the node's MOP service client when the node is configured as a boot server. By default, it is the OpenVMS Cluster node name (for example, the SCS node name). This name does not need to be the same as the OpenVMS Cluster node name.

For systems running TCP/IP or the LANCP Utility for satellite booting, or both

Computer's SCS node name (SCSNODE) and SCS system ID (SCSSYSTEMID)

These prompts are described in Section 4.2.3, “Information Required”. If a system is running TCP/IP, the procedure does not ask for a TCP/IP host name because a cluster node name (SCSNODE) does not have to match a TCP/IP host name. The TCP/IP host name might be longer than six characters, whereas the SCSNODE name must be no more than six characters. Note that if the system is running both DECnet and IP, then the procedure uses the DECnet defaults.

For LAN configurations

Cluster group number and password

This information is requested only when the CHANGE option is chosen. See Section 2.5, “OpenVMS Cluster Membership” for information about assigning cluster group numbers and passwords.

Satellite's LAN hardware address

Address has the form xx-xx-xx-xx-xx-xx. You must include the hyphens when you specify a hardware address. For getting the hardware address, execute the following command at the satellite's console:

On Integrity servers:
Shell> lanaddress
On Alpha systems:
>>> SHOW NETWORK

These commands display the hardware address of the LAN devices that can be used for satellite booting. Note that you can also use the SHOW CONFIG command at LANCP.

For IP configurations

UDP port number

UDP port number is the port number used for cluster communication. UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number or this port number is used by any other application.

IP multicast address

Enter the IP multicast address for cluster, if IP multicasting is enabled. By default, the IP multicast address is selected from the administratively scoped IP multicast address range of 239.242.x.y. The last two octets x and y are generated based on the cluster group number. For example, if the cluster group number is 1985, the multicast is calculated as follows:
X= 1985/256 Y= 1985 - (256 *x)

The system administrator can override the default multicast address with a unique address for their environment.

IP unicast address

If the node that is configured uses IP unicast to discover a remote note, you need the IP unicast address of the existing members or any new member in the cluster.

IP address

It is the IP address of the local system from where the cluster is configured.

Gateway and Network mask address

In the configuration option, select option 4 to add the TCP/IP gateway and network mask address to the cluster over IP database.

IP interface address

In the configuration option for the selected address, select option 4 to add to the cluster over IP database. The interface information along with the default route is entered in the TCPIP$CLUSTER.DAT as shown in the following example:

interface=IE0,EIA0,10.0.1.2,255.255.255.0

default_route=10.0.1.1

IP interface address for satellite booting

To select the IP interface to be used for satellite booting.

For Alpha systems:
  • Execute the following command at the satellite's console:
    >>> SHOW DEVICE    

    From the output, the LAN interface will be EIA0 on which IP address will be configured and used for cluster configuration.

For Integrity server systems:
  • The IP interface name will either start from 'EI' or 'EW'. If it is the first interface, it will be EIA0 or EWA0. Note the mac address of the interface that you want to use from the Shell prompt.

    To get the interface information on Integrity servers, execute the following command on EFI Shell:
    Shell> lanaddress

    Assume the interface which is active is EIA0. Configure the satellite with EIA0, if it does not boot with EIA0, try with EWA0 subsequently.

8.1.3. Invoking the Procedure

Once you have made the necessary preparations, you can invoke the cluster configuration procedure to configure your OpenVMS Cluster system. Log in to the system manager account and make sure your default is SYS$MANAGER. Then, invoke the procedure at the DCL command prompt as follows:
$ @CLUSTER_CONFIG_LAN
or
$ @CLUSTER_CONFIG

Caution: Do not invoke multiple sessions simultaneously. You can run only one cluster configuration session at a time.

Once invoked, both procedures display the following information and menu. (The only difference between CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM at this point is the command procedure name that is displayed.) Depending on the menu option you select, the procedure interactively requests configuration information from you. (Predetermine your answers as described in Table 8.3, “Data Requested by CLUSTER_CONFIG_LAN.COM and CLUSTER_CONFIG.COM”).
               Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet Phase IV is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    CALVIN is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for CALVIN.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]:
.
.
   .
   .
   .

This chapter contains a number of sample sessions showing how to run the cluster configuration procedures. Although the CLUSTER_CONFIG_LAN.COM and the CLUSTER_CONFIG.COM procedure function the same for Integrity server systems and Alpha systems, the questions and format may appear slightly different according to the type of computer system.

8.2. Adding Computers

In most cases, you invoke either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM on an active OpenVMS Cluster computer and select the ADD function to enable a computer as an OpenVMS Cluster member. However, in some circumstances, you may need to perform extra steps to add computers. Use the information in Table 8.4, “Preparing to Add Computers to an OpenVMS Cluster” to determine your actions.
Table 8.4. Preparing to Add Computers to an OpenVMS Cluster
IF...THEN...

You are adding your first satellite node to the OpenVMS Cluster.

Follow these steps:
  1. Log in to the computer that will be enabled as the cluster boot server.

  2. Invoke the cluster configuration procedure, and execute the CHANGE function described in Section 8.4, “Changing Computer Characteristics” to enable the local computer as a boot server.

  3. After the CHANGE function completes, execute the ADD function to add satellites to the cluster.

The cluster uses DECdtm services.

You must create a transaction log for the computer when you have configured it into your cluster. For step-by-step instructions on how to do this, see the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual.

You add a CI connected computer that boots from a cluster common system disk.

You must create a new default bootstrap command procedure for the computer before booting it into the cluster. For instructions, refer to your computer-specific installation and operations guide.

You are adding computers to a cluster with more than one common system disk.

You must use a different device name for each system disk on which computers are added. For this reason, the cluster configuration procedure supplies as a default device name the logical volume name (for example, DISK$MARS_SYS1) of SYS$SYSDEVICE: on the local system.

Using different device names ensures that each computer added has a unique root directory specification, even if the system disks contain roots with the same name—for example, DISK$MARS_SYS1:[SYS10] and DISK$MARS_SYS2:[SYS10].

You add a voting member to the cluster.

You must, after the ADD function completes, reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”.

Caution: If either the local or the new computer fails before the ADD function completes, you must, after normal conditions are restored, perform the REMOVE option to erase any invalid data and then restart the ADD option. Section 8.3, “Removing Computers” describes the REMOVE option.

8.2.1. Controlling Conversational Bootstrap Operations

When you add a satellite to the cluster using either cluster configuration command procedure, the procedure asks whether you want to allow conversational bootstrap operations for the satellite (default is No).

If you select the default, the NISCS_CONV_BOOT system parameter in the satellite's system parameter file remains set to 0 to disable such operations. The parameter file (IA64VMSSYS.PAR for Integrity servers or ALPHAVMSSYS.PAR for Alpha systems) resides in the satellite's root directory on a boot server's system disk (device:[SYS x.SYSEXE]). You can enable conversational bootstrap operations for a given satellite at any time by setting this parameter to 1.

Example:

To enable such operations for an OpenVMS Alpha satellite booted from root 10 on device $1$DGA11, you would proceed as follows:

Step

Action

1

Log in as system manager on the boot server.

2

On Integrity servers or Alpha systems, invoke the System Generation utility (SYSGEN) and enter the following commands:
$ RUN SYS$SYSTEM:SYSGEN
SYSGEN> USE $1$DGA11:[SYS10.SYSEXE]ALPHAVMSSYS.PAR
SYSGEN> SET NISCS_CONV_BOOT 1
SYSGEN> WRITE $1$DGA11:[SYS10.SYSEXE]ALPHAVMSSYS.PAR
SYSGEN> EXIT
$

3

Modify the satellite's MODPARAMS.DAT file so that NISCS_CONV_BOOT is set to 1.

8.2.2. Common AUTOGEN Parameter Files

When adding a node or a satellite to an OpenVMS Cluster, the cluster configuration command procedure adds one of the following lines in the MODPARAMS.DAT file:

WHEN the node being added is a...

THEN...

Satellite node

The following line is added to the MODPARAMS.DAT file:
AGEN$INCLUDE_PARAMS SYS$MANAGER:AGEN$NEW_SATELLITE_DEFAULTS.DAT

Nonsatellite node

The following line is added to the MODPARAMS.DAT file:
AGEN$INCLUDE_PARAMS SYS$MANAGER:AGEN$NEW_NODE_DEFAULTS.DAT

The AGEN$NEW_SATELLITE_DEFAULTS.DAT and AGEN$NEW_NODE_DEFAULTS.DAT files hold AUTOGEN parameter settings that are common to all satellite nodes or nonsatellite nodes in the cluster. Use of these files simplifies system management, because you can maintain common system parameters in either one or both of these files. When adding or changing the common parameters, this eliminates the need to make modifications in the MODPARAMS.DAT files located on every node in the cluster.

Initially, these files contain no parameter settings. You edit the AGEN$NEW_SATELLITE_DEFAULTS.DAT and AGEN$NEW_NODE_DEFAULTS.DAT files, as appropriate, to add, modify, or edit system parameters. For example, you might edit the AGEN$NEW_SATELLITE_DEFAULTS.DAT file to set the MIN_GBLPAGECNT parameter to 5000. AUTOGEN makes the MIN_GBLPAGECNT parameter and all other parameter settings in the AGEN$NEW_SATELLITE_DEFAULTS.DAT file common to all satellite nodes in the cluster.

AUTOGEN uses the parameter settings in the AGEN$NEW_SATELLITE_DEFAULTS.DAT or AGEN$NEW_NODE_DEFAULTS.DAT files the first time it is run, and with every subsequent execution.

8.2.3. Examples

Examples 8.1, 8.2, and 8.3 describes the use of CLUSTER_CONFIG_LAN.COM on BHAGAT to add, respectively, a boot server running DECnet for OpenVMS, a boot server running DECnet–Plus, and a satellite node.

This section also illustrates the use of CLUSTER_CONFIG_LAN.COM to create and configure a two node cluster using IPCI, to add a new node to an IPCI cluster, to add a new node to an IPCI cluster with a Shared System Disk, and to add an Integrity server satellite node to an IPCI cluster.
Example 8.1. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Add a Computer as a Boot Server
$  @CLUSTER_CONFIG_LAN.COM
               Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 1

    This ADD function will add a new IA64 node to the cluster.

  WARNING: If the node being added is a voting member, EXPECTED_VOTES for
           every cluster member must be adjusted.  For complete instructions
           check the section on configuring a cluster in the "OpenVMS Cluster
           Systems" manual.

  CAUTION: If this cluster is running with multiple system disks and
           common system files will be used, please, do not proceed
           unless appropriate logical names are defined for cluster
           common files in SYLOGICALS.COM. For instructions, refer to
           the "OpenVMS Cluster Systems" manual.

           If this cluster will run IPCI, then TCP/IP installed on the system
           should be version 5.7 and above or else IPCI configuration will be
           aborted.

Do you want to continue [Y]?[Return]
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y

Will the node be a satellite [Y]? N
What is the node's SCS node name? MOON
What is the node's SCSSYSTEMID number? 24.123
    NOTE: 24.123 equates to an SCSSYSTEMID of 24699
Will MOON be a boot server [Y]? [Return]

        TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes.

   Use SYS$MANAGER:TCPIP$CONFIG.COM on MOON to enable BOOTP and TFTP service
        after MOON has booted into the cluster.

   This procedure will now ask you for the device name of MOON's system root.
    The default device name (DISK$BHAGAT_831H1:) is the logical volume name of
    SYS$SYSDEVICE:.

What is the device name for MOON's system root [default DISK$BHAGAT_831H1:]?
What is the name of MOON's system root [SYS1]? [Return]
    Creating directory tree SYS1 ...
    System root SYS1 created
ENABLE IP for cluster communications (Y/N)? N

  CAUTION: If you do not define port allocation classes later in this
           procedure for shared SCSI buses, all nodes sharing a SCSI bus
           must have the same non-zero ALLOCLASS value. If multiple
           nodes connect to a shared SCSI bus without the same allocation
           class for the bus, system booting will halt due to the error or
           IO AUTOCONFIGURE after boot will keep the bus offline.

  WARNING: If BHAGAT is sharing the same SCSI bus with MOON, then BHAGAT's
           ALLOCLASS parameter or port allocation class for the shared bus
           must be changed from 0 to the same non-zero value that will be
           entered for MOON. Use the CHANGE option of
           CLUSTER_CONFIG_LAN.COM to change BHAGAT's ALLOCLASS
           parameter before MOON is booted.

Enter a value for MOON's ALLOCLASS parameter [1]: [Return]
Does this cluster contain a quorum disk [N]? [Return]
Size of pagefile for MOON [RETURN for AUTOGEN sizing]? [Return]

    A temporary pagefile will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary pagefile [10000]? [Return]
Size of swap file for MOON [RETURN for AUTOGEN sizing]? [Return]

    A temporary swap file will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary swap file [8000]? [Return]
    Each shared SCSI bus must have a positive allocation class value. A shared
    bus uses a PK adapter. A private bus may use: PK, DR, DV, DQ.

    When adding a node with SCSI-based cluster communications, the shared
    SCSI port allocation classes may be established in SYS$DEVICES.DAT.
    Otherwise, the system's disk allocation class will apply.

    A private SCSI bus need not have an entry in SYS$DEVICES.DAT. If it has an
    entry, its entry may assign any legitimate port allocation class value:

       n   where n = a positive integer, 1 to 32767 inclusive
       0   no port allocation class and disk allocation class does not apply
      -1   system's disk allocation class applies (system parameter ALLOCLASS)

    When modifying port allocation classes, SYS$DEVICES.DAT must be updated
    for all affected nodes, and then all affected nodes must be rebooted.
    The following dialog will update SYS$DEVICES.DAT on MOON.

Enter [RETURN] to continue:

    There are currently no entries in SYS$DEVICES.DAT for MOON.
    After the next boot, any SCSI controller on MOON will use
    MOON's disk allocation class.


Assign port allocation class to which adapter [RETURN for none]: [Return]
Will a disk local only to MOON (and not accessible at this time to BHAGAT)
be used for paging and swapping (Y/N)? N

    If you specify a device other than DISK$BHAGAT_831H1: for MOON's
    page and swap files, this procedure will create PAGEFILE_MOON.SYS
    and SWAPFILE_MOON.SYS in the <SYSEXE> directory on the device you
    specify.

What is the device name for the page and swap files [DISK$BHAGAT_831H1:]?
%SYSGEN-I-CREATED, BHAGAT$DKA100:<SYS1.SYSEXE>PAGEFILE.SYS;1 created
%SYSGEN-I-CREATED, BHAGAT$DKA100:<SYS1.SYSEXE>SWAPFILE.SYS;1 created
    The configuration procedure has completed successfully.

    MOON has been configured to join the cluster.
    The first time MOON boots, AUTOGEN.COM will run automatically. 

Example 8.2. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Add a Computer Running DECnet–Plus
$ @CLUSTER_CONFIG.COM
               Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an Alpha System

    DECnet-Plus is installed on this node.
    Alpha satellites will use LANCP, not DECnet, for MOP downline loading.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BISMIL is an Alpha system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an Alpha node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BISMIL.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 1

    This ADD function will add a new Alpha node to the cluster.

  WARNING: If the node being added is a voting member, EXPECTED_VOTES for
           every cluster member must be adjusted.  For complete instructions
           check the section on configuring a cluster in the "OpenVMS Cluster
           Systems" manual.

  CAUTION: If this cluster is running with multiple system disks and
           common system files will be used, please, do not proceed
           unless appropriate logical names are defined for cluster
           common files in SYLOGICALS.COM. For instructions, refer to
           the "OpenVMS Cluster Systems" manual.

           If this cluster will run IPCI, then TCP/IP installed on the system
           should be version 5.7 and above or else IPCI configuration will be
           aborted.

Do you want to continue [Y]? [Return]
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y

Will the node be a satellite [Y]? N
What is the node's SCS node name? MOON

    DECnet is running on this node. Even though you are configuring a LAN-
    based cluster, the DECnet database will provide some information and
    may be updated.

What is the node's DECnet full name? local:.MOON
Do you want to define a DECnet synonym [Y]? N
What is the MOP service client name for this node [MOON]? VENUS
What is the node's SCSSYSTEMID number? 24.123
    NOTE: 24.123 equates to an SCSSYSTEMID of 24699
Will MOON run DECnet [Y]? [Return]

    Note:
        This procedure will not update any network databases
        with information about MOON.  You must do that
        yourself.

Will MOON be a boot server [Y]? [Return]

   This procedure will now ask you for the device name of MOON's system root.
    The default device name (DISK$ALPHA732:) is the logical volume name of
    SYS$SYSDEVICE:.

What is the device name for MOON's system root [default DISK$ALPHA732:]?
What is the name of MOON's system root [SYS1]? [Return]
    Creating directory tree SYS1 ...
    System root SYS1 created
ENABLE IP for cluster communications (Y/N)? N

  CAUTION: If you do not define port allocation classes later in this
           procedure for shared SCSI buses, all nodes sharing a SCSI bus
           must have the same non-zero ALLOCLASS value. If multiple
           nodes connect to a shared SCSI bus without the same allocation
           class for the bus, system booting will halt due to the error or
           IO AUTOCONFIGURE after boot will keep the bus offline.

  WARNING: If BISMIL is sharing the same SCSI bus with MOON, then BISMIL's
           ALLOCLASS parameter or port allocation class for the shared bus
           must be changed from 0 to the same non-zero value that will be
           entered for MOON. Use the CHANGE option of
           CLUSTER_CONFIG_LAN.COM to change BISMIL's ALLOCLASS
           parameter before MOON is booted.

Enter a value for MOON's ALLOCLASS parameter [1]: [Return]
Does this cluster contain a quorum disk [N]? [Return]
Size of pagefile for MOON [RETURN for AUTOGEN sizing]? [Return]

    A temporary pagefile will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary pagefile [10000]? [Return]
Size of swap file for MOON [RETURN for AUTOGEN sizing]? [Return]

    A temporary swap file will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary swap file [8000]? [Return]
    Each shared SCSI bus must have a positive allocation class value. A shared
    bus uses a PK adapter. A private bus may use: PK, DR, DV, DQ.

    When adding a node with SCSI-based cluster communications, the shared
    SCSI port allocation classes may be established in SYS$DEVICES.DAT.
    Otherwise, the system's disk allocation class will apply.

    A private SCSI bus need not have an entry in SYS$DEVICES.DAT. If it has an
    entry, its entry may assign any legitimate port allocation class value:

       n   where n = a positive integer, 1 to 32767 inclusive
       0   no port allocation class and disk allocation class does not apply
      -1   system's disk allocation class applies (system parameter ALLOCLASS)

    When modifying port allocation classes, SYS$DEVICES.DAT must be updated
    for all affected nodes, and then all affected nodes must be rebooted.
    The following dialog will update SYS$DEVICES.DAT on MOON.

Enter [RETURN] to continue: [Return]

    There are currently no entries in SYS$DEVICES.DAT for MOON.
    After the next boot, any SCSI controller on MOON will use
    MOON's disk allocation class.


Assign port allocation class to which adapter [RETURN for none]:
Will a local (non-HSx) disk on MOON and not on a hierarchical storage
controller be used for paging and swapping (Y/N)? N

    If you specify a device other than DISK$ALPHA732: for MOON's
    page and swap files, this procedure will create PAGEFILE_MOON.SYS
    and SWAPFILE_MOON.SYS in the <SYSEXE> directory on the device you
    specify.

What is the device name for the page and swap files [DISK$ALPHA732:]?
%SYSGEN-I-CREATED, BISMIL$DKB100:<SYS1.SYSEXE>PAGEFILE.SYS;1 created
%SYSGEN-I-CREATED, BISMIL$DKB100:<SYS1.SYSEXE>SWAPFILE.SYS;1 created
    The configuration procedure has completed successfully.

    MOON has been configured to join the cluster.

    Before booting MOON, you must create a new default
    bootstrap command procedure for MOON. For instructions,
    see your processor-specific installation and operations guide.

    The first time MOON boots, NET$CONFIGURE.COM and
    AUTOGEN.COM will run automatically.

    The following parameters have been set for MOON:

                  VOTES = 1
                  QDSKVOTES = 1

    After MOON has booted into the cluster, you must increment
    the value for EXPECTED_VOTES in every cluster member's
    MODPARAMS.DAT. You must then reconfigure the cluster, using the
    procedure described in the "OpenVMS Cluster Systems" manual. 

Example 8.3. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Add a Satellite with Local Page and Swap Files
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 1

    This ADD function will add a new IA64 node to the cluster.

  WARNING: If the node being added is a voting member, EXPECTED_VOTES for
           every cluster member must be adjusted.  For complete instructions
           check the section on configuring a cluster in the "OpenVMS Cluster
           Systems" manual.

  CAUTION: If this cluster is running with multiple system disks and
           common system files will be used, please, do not proceed
           unless appropriate logical names are defined for cluster
           common files in SYLOGICALS.COM. For instructions, refer to
           the "OpenVMS Cluster Systems" manual.

           If this cluster will run IPCI, then TCP/IP installed on the system
           should be version 5.7 and above or else IPCI configuration will be
           aborted.

Do you want to continue [Y]? [Return]
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N

Will the node be a satellite [Y]? [Return]
What is the node's SCS node name? GOMTHI

    DECnet is running on this node. Even though you are configuring a LAN-
    based cluster, the DECnet database will provide some information and
    may be updated.

What is the node's DECnet full name? local:.GOMTHI
Do you want to define a DECnet synonym [Y]? N
What is the node's SCSSYSTEMID number? 25.171
    NOTE: 25.171 equates to an SCSSYSTEMID of 25771

  WARNING:
    The DECnet databases on BHAGAT will not be updated with
    information on GOMTHI. You must see to it that network
    databases on this and all other cluster members are updated.
    For help, refer to the "OpenVMS Cluster Systems" manual.

Does GOMTHI need to be registered in the DECnet namespace [N]?[Return]
What is the Cluster Alias full name? [Return]
Will GOMTHI run DECnet [Y]? [Return]

   This procedure will now ask you for the device name of GOMTHI's system root.
    The default device name (DISK$BHAGAT_SYS:) is the logical volume name of
    SYS$SYSDEVICE:.

What is the device name for GOMTHI's system root [default DISK$BHAGAT_SYS:]?
What is the name of GOMTHI's system root [SYS10]? [Return]
What is GOMTHI's LAN adapter hardware address? 00-30-6E-4C-BB-1A
What is GOMTHI's TCP/IP address? 16.181.160.129
Would you like GOMTHI added as a TCP/IP host shortcut for 16.181.160.129 [Y]? [Return]
What is GOMTHI's TCP/IP gateway or gateways (leave blank if none)? 16.181.160.1
What is GOMTHI's TCP/IP network mask [255.255.255.0]?  255.255.252.0

  NOTE:  Make sure to set the VMS_FLAGS console variable
         to 0,200000 on node GOMTHI so it will use
         the memory-disk method to boot as a satellite.
         The command to update this variable from the
         console EFI shell of GOMTHI is:
           set vms_flags "0,200000"

Allow conversational bootstraps on GOMTHI [N]? [Return]

    The following workstation windowing options are available:

       1. No workstation software
       2. DECwindows Workstation Software

Enter choice [1]: [Return]
    Creating directory tree SYS10 ...
    System root SYS10 created
ENABLE IP for cluster communications (Y/N)? N
Will GOMTHI be a disk server [N]? Y
Enter a value for GOMTHI's ALLOCLASS parameter [0]: [Return]
    Updating BOOTP database with satellite information for GOMTHI...
Size of pagefile for GOMTHI [RETURN for AUTOGEN sizing]? [Return]

    A temporary pagefile will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary pagefile [10000]? [Return]
Size of swap file for GOMTHI [RETURN for AUTOGEN sizing]? [Return]

    A temporary swap file will be created until resizing by AUTOGEN. The
    default size below is arbitrary and may or may not be appropriate.

Size of temporary swap file [8000]? [Return]

   NOTE:  IA64 satellite node GOMTHI requires DOSD if capturing the
          system state in a dumpfile is desired after a system crash.

Will a local disk on GOMTHI be used for paging and swapping (Y/N)? Y

    This procedure will now wait until GOMTHI is a member of
    the cluster.  Once GOMTHI joins the cluster, this procedure
    will ask you which local disk it can use for paging and swapping.

    Please boot GOMTHI now. Make sure the default boot device is
    set to the appropriate clustered-disk access path: LAN device for
    satellite nodes; or shared-bus (CI/DSSI/SCSI/FC) disk device.  See
    the hardware user manual or the console help command for instructions
    to do this.

    Waiting for GOMTHI to boot...
    Waiting for GOMTHI to boot...
    Waiting for GOMTHI to boot...
    Waiting for GOMTHI to boot...
    Waiting for GOMTHI to boot...

    Node GOMTHI is now a cluster member. This procedure will pause
    for up to 4 minutes, while attempting to detect local disks on
    GOMTHI, to use for paging and swapping.


    The local disks on GOMTHI are:

Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label        Blocks Count Cnt
GOMTHI$DQA0:            Online               0
GOMTHI$DKA0:            Online               0
GOMTHI$DKA100:          Online               0
GOMTHI$DKB200:          Online               0

    If the paging and swapping disk you plan to use is not displayed,
    it may not yet be configured.  Please wait a few moments and hit
    a carriage return for an updated display.

Which disk can be used for paging and swapping? GOMTHI$DKA100:
May this procedure INITIALIZE GOMTHI$DKA100 [Y]? N

    In order to ensure that this disk has a unique volume name this
    procedure wishes to change its name from [GOMTHI_831H1] to
    [GOMTHI_25771].  If the satellite being added may also be booted
    standalone and refers to this disk by name you may retain the old volume
    name if there are no other disks with the same name in this cluster.

May the volume name of this disk be changed to GOMTHI_25771 [Y]? N
%DELETE-W-SEARCHFAIL, error searching for SYS$COMMON:[SYSMGR]CLU2020042F.TMP1;*
-RMS-E-FNF, file not found
    Mounting GOMTHI$DKA100...

What is the file specification for the pagefile on
GOMTHI$DKA100: [ <SYSEXE>PAGEFILE.SYS ]? [Return]
%CREATE-I-CREATED, GOMTHI$DKA100:<SYSEXE> created
%SYSGEN-I-CREATED, GOMTHI$DKA100:<SYSEXE>PAGEFILE.SYS;1 created

What is the file specification for the swapfile on
GOMTHI$DKA100: [ <SYSEXE>SWAPFILE.SYS ]? [Return]
%SYSGEN-I-CREATED, GOMTHI$DKA100:<SYSEXE>SWAPFILE.SYS;1 created

    SATELLITE_PAGE.COM and INSTALL_PAGE.COM will now be created for local
    page/swap disk/file installation.

             ****** ! SHADOWED PAGE or SWAP DISK WARNING !  ******
             ****  Edit these procedures to include any       ****
             ****  local configuration commands necessary for ****
             ****  shadowed disk mounting, prior to reboot.   ****
             *****************************************************

    AUTOGEN will now reconfigure and reboot GOMTHI automatically.
    These operations will complete in a few minutes, and a
    completion message will be displayed at your terminal.

    Waiting for GOMTHI to reboot...
    Waiting for GOMTHI to reboot...

    The configuration procedure has completed successfully.

8.2.3.1. Creating and Configuring a Two-Node Cluster Using IP

Cluster over IP can be used to create and configure a two-node cluster. Node ORCHID is a standalone node at SITE A and node TULIP at SITE B, is already a member (the only member) of the cluster. In this scenario, Cluster over IP is not configured in TULIP. SITE A and SITE B can be in the same or different LAN, building or any other geographical location. It is required to have IP connectivity between SITE A and SITE B and should be within the supported inter site distance.

Step 1. Configuring Node TULIP to Enable Cluster over IP

To configure the node TULIP (OpenVMS Alpha node) for enabling the Cluster over IP feature, execute the CLUSTER_CONFIG_LAN.COM procedure on node TULIP and select the appropriate option as illustrated:
Example 8.4. Configuring Node TULIP to Enable Cluster over IP
TULIP$@SYS$MANAGER:CLUSTER_CONFIG_LAN
                 Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an Alpha System

DECnet-Plus is installed on this node.
Alpha satellites will use LANCP, not DECnet, for MOP downline loading.

Enter a "?" for help at any prompt. If you are familiar with the execution of
this procedure, you may want to mute extra notes and
explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

TULIP is an Alpha system and currently a member of a cluster so the following
functions can be performed:

MAIN Menu

   1. ADD an Alpha node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for TULIP.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3 1

CHANGE Menu

   1. Enable TULIP as a boot server.
   2. Disable TULIP as a boot server.
   3. Enable a quorum disk for TULIP
   4. Disable a quorum disk for TULIP.
   5. Enable TULIP as a disk server.
   6. Disable TULIP as a disk server.
   7. Change TULIP's ALLOCLASS value.
   8. Enable TULIP as a tape server.
   9. Disable TULIP as a tape server.
  10. Change TULIP's TAPE_ALLOCLASS value.
  11. Change an Alpha satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on TULIP.
  13. Disable Cluster Communication using IP on TULIP.
  14. Enable the LAN for cluster communications on TULIP.
  15. Disable the LAN for cluster communications on TULIP.
  16. Enable Memory Channel for cluster communications on TULIP.
  17. Disable Memory Channel for cluster communications on TULIP.
  18. Change TULIP's shared SCSI port allocation class value.
  19. Return to MAIN menu.

Enter choice [19]: 12 2
ENABLE IP for cluster communications (Y/N)? Y 3
UDP port number to be used for Cluster Communication over IP[49152]?Y 4
Enable IP multicast for cluster communication(Y/N)[Y]? Y 5
What is the IP multicast address[239.242.7.193]? [Return] 6
What is the TTL (time to live) value for IP multicast packets [32]?  [Return] 7
Do you want to enter unicast address(es)(Y/N)[Y]?  [Return] 8
What is the unicast address [Press Enter to end the list]? 10.0.1.2 9
What is the unicast address[Press Enter to end the list]?  [Return] 10

   *****************************************************************
        Cluster Communications  over IP  has  been  enabled.  Now
        CLUSTER_CONFIG_LAN will run the  SYS$MANAGER:TCPIP$CONFIG
        procedure. Please select the IP interfaces to be used for
        Cluster Communications  over IP  (IPCI). This can be done
        selecting "Core Environment"  option from the main menu
        followed by the "Interfaces"  option.  You may also use
        this opportunity to configure other aspects.
   ****************************************************************

Press Enter to continue.

       Checking TCP/IP Services for OpenVMS configuration database files.

        TCP/IP Services for OpenVMS Configuration Menu

Configuration options:

                 1  -  Core environment
                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option: 1 11

       TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option: 2 12

TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=TULIP, Active=TULIP

 Configuration options:

                 0  -  Set The Target Node (Current Node: TULIP)
                 1  -  IE0 Menu (EIA0: TwistedPair 100mbps)
                 2  -  10.0.2.2/23    TULIP                  Configured,Active
                 3  -  IE1 Menu (EIB0: TwistedPair 100mbps)
                 4  -  10.0.2.224/23   *noname*              Configured,Active
                 I  -  Information about your configuration
                [E] -  Exit menu

Enter configuration option: 2 13


  TCP/IP Services for OpenVMS Address Configuration Menu (Node: TULIP)

      IE0 10.0.2.2/23 TULIP Configured, Active IE0

 Configuration options

                 1  - Change address
                 2  - Set "TULIP" as the default hostname
                 3  - Delete from configuration database
                 4  - Add to IPCI database
                 5  - Deactivate from live system
                 6  - Add standby aliases to configuration database (for failSAFE IP)
                [E] - Exit menu

Enter configuration option: 4 14
Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT;

Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT;
Added address IE1:10.0.2.2 to IPCI database

   TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=TULIP, Active=TULIP

 Configuration options:

                 0  -  Set The Target Node (Current Node:tulip)
                 1  -  IE0 Menu (EIA0: TwistedPair 100mbps)
                 2  -  10.0.2.2/23    TULIP                  Configured,IPCI,Active
                 3  -  IE1 Menu (EIB0: TwistedPair 100mbps)
                 4  -  10.0.2.224/23   *noname*              Configured,Active
                 I  -  Information about your configuration
                [E]-  Exit menu

Enter configuration option: E 15

      TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option: E

        TCP/IP Services for OpenVMS Configuration Menu

        Configuration options:

                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option: E
    The configuration procedure has completed successfully.

    Tulip has been enabled for IP communications  16
    Please run AUTOGEN to reboot TULIP:

TULIP$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT 17

1

TULIP is a single-member cluster without Cluster over IP enabled. The cluster member characteristics can be changed to enable Cluster over IP for this node by selecting option 3.

2

Select option 12 to enable cluster over IP. By selecting this option, the SYSGEN parameter, NISCS_USE_UDP is set to 1, which enables the PEDRIVER to use IP for cluster communication. This requires reboot of the node. If LAN is not already selected as the cluster interconnect, this option sets NISCS_LOAD_PEA0 to 1 to load PEDRIVER during the next reboot.

3

Enable IP for cluster communication.

4

You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator, if IP multicasting is enabled in your environment.

5

You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator, if IP multicasting is enabled in your environment.

6

Enter the IP multicast address for the cluster, if IP multicasting is enabled. By default, the IP multicast address is selected from the administratively scoped IP multicast address ranging from 239.242.x.y. The last two octets x and y are generated based on the cluster group number. In the above example cluster group number is 1985 and can be calculated as follows:
X= 1985/256
Y= 1985 - (256 *x)
The system administrator can override the default multicast address by a unique address for their environment.

7

TTL is the time to live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets.

8

Enter "Yes" to enter the IP unicast addresses for the remote nodes of the cluster, which are not reachable using IP multicast address.

9

In this example, 10.0.1.2 is the IP unicast address for the node ORCHID. Although, the IP multicast is selected, ORCHID's IP address is entered because the IP multicast connectivity between SITE A and SITE B is presumed to be not existing in this example. Note: Enter the list of IP address of the cluster. All the information entered in[4],[6],[7] and [9] are entered into the SYS$SYSTEM:PE$IP_CONFIG.DAT file. The PE$IP_CONFIG.DAT file is generated as shown in the following example. Also, to allow the remote node to join the cluster, the Unicast list in the PE$IP_CONFIG.DAT on the local node must contain the IP address of the remote node. In this example, TULIP must have ORCHID's IP address and ORCHID must have TULIP's IP address.
! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL-2008
14:14:06.57

multicast_address=239.242.7.193

ttl=32

udp_port=49152

unicast=10.0.1.2

10

Press Return after entering the Unicast list.

11

CLUSTER_CONFIG_LAN.COM invokes TCPIP$CONFIG.COM to configure the IP interfaces used for cluster communication. Select the core environment option. Assuming that TCP/IP is already configured, the node can be pinged from outside the subnet.

12

Select the interface option from the core environment menu.

13

Select the appropriate interface for cluster communication. In this example, option 2 is selected.

14

In the configuration option for the selected address, select option 4 to add to IPCI database. The interface information along with the default route is entered in the TCPIP$CLUSTER.DAT as shown:
interface=IE0,EIA0,10.0.2.2,255.255.254.0

default_route=10.0.2.1

15

Exit from the TCP/IP configuration procedure, which returns to CLUSTER_CONFIG_LAN.COM.

16

Proceed with cluster configuration.

17

After rebooting the system, run AUTOGEN. PEDRIVER in ORCHID will start using IP in addition to LAN for cluster communication and must be able to join TULIP.

Step 2. Configuring Node ORCHID as a Cluster Member

To configure ORCHID with Cluster over IP enabled, execute CLUSTER_CONFIG_LAN.COM on node ORCHID and select the appropriate option as shown in the following example:
Example 8.5. Configuring Node ORCHID to Enable Cluster over IP
ORCHID$ @SYS$MANAGER:CLUSTER_CONFIG_LAN.COM
                 Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System
    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline
    loading.
     TCP/IP is installed and running on this node.

Enter a "?" for help at any prompt.  If you are familiar with the execution of this
procedure, you may want to mute extra notes and
explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

This IA64 node is not currently a cluster member.

MAIN Menu

   1. ADD ORCHID to existing cluster, or form a new cluster.
   2. MAKE a directory structure for a new root on a system disk.
   3. DELETE a root from a system disk.
   4. EXIT from this procedure.

Enter choice [4]: 1 1
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N
What is the node's SCS node name? ORCHID
    IA64 node, using LAN/IP for cluster communications.  PEDRIVER will be loaded.
   No other cluster interconnects are supported for IA64 nodes.
Enter this cluster's group number: 1985
Enter this cluster's password:
Re-enter this cluster's password for verification:

ENABLE IP for cluster communications (Y/N)? Y 2
UDP port number to be used for Cluster Communication over IP[49152]?[Return] 3
Enable IP multicast for cluster communication(Y/N)[Y]? Y [Return] 4
What is IP the multicast address[239.242.7.193]? [Return] 5
What is the TTL (time to live) value for IP multicast packets [32]? [Return] 6
Do you want to enter unicast address(es)(Y/N)[Y]?[Return] 7
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2 8
What is the unicast address[Press [RETURN] to end the list]?[Return] 9


                 ********************************************************************
                 Cluster Communications over IP has been enabled.  Now
                 CLUSTER_CONFIG_LAN will run the SYS$MANAGER:TCPIP$CONFIG procedure.
                 Pleas select the IP interfaces to be used for Cluster Communications
                 over IP (IPCI). This can be done selecting "Core Environment" option
                 from the main menu followed by the "Interfaces" option. You may
                 also use this opportunity to configure other aspects.
                 *********************************************************************

Press Return to continue ...

TCP/IP Network Configuration Procedure

        This procedure helps you define the parameters required
        to run TCP/IP Services for OpenVMS on this system.

%TCPIP-I-IPCI, TCP/IP Configuration is limited to IPCI.
-TCPIP-I-IPCI, Rerun TCPIP$CONFIG after joining the cluster.


    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=Not Configured, Active=nodeg

 Configuration options:

   0  -  Set The Target Node (Current Node: ORCHID)
   1  -  IE0 Menu (EIA0: TwistedPair 100mbps)
   2  -  IE1 Menu (EIB0: TwistedPair 100mbps)
  [E] -  Exit menu

Enter configuration option: 1 10

                    * IPCI Address Configuration *

Only IPCI addresses can be configured in the current environment.
After configuring your IPCI address(es) it will be necessary to
run TCPIP$CONFIG once your node has joined the cluster.


    IPv4 Address may be entered with CIDR bits suffix.
    E.g. For a 16-bit netmask enter 10.0.1.1/16

Enter IPv4 Address []:10.0.1.2/24 11
Default netmask calculated from class of IP address: 255.0.0.0

    IPv4 Netmask may be entered in dotted decimal notation,
    (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16)

Enter Netmask or CIDR bits [255.0.0.0]: 255.255.254.0 12

Requested configuration:

      Node     : ORCHID
      Interface: IE0
      IPCI     : Yes
      Address  : 10.0.1.2/23
      Netmask  : 255.255.254.0 (CIDR bits: 23)

* Is this correct [YES]:
Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT; 13


    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=Not Configured, Active=ORCHID

 Configuration options:

 0  -  Set The Target Node (Current Node: ORCHID)
 1  -  IE0 Menu (EIA0: TwistedPair 100mbps)
 2  - 10.0.1.2 /23   *noname*              IPCI
 3  -  IE1 Menu (EIB0: TwistedPair 100mbps)
[E] -  Exit menu

Enter configuration option: E 14
Enter your Default Gateway address []: 10.0.1.1 15
* The default gateway will be: 10.0.1.1.  Correct [NO]: YES
Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT;
TCPIP-I-IPCIDONE, Finished configuring IPCI address information 16
Will ORCHID be a boot server [Y]? [Return] 17

       TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes.

        Use SYS$MANAGER:TCPIP$CONFIG.COM on ORCHID to enable BOOTP and TFTP
        services after ORCHID has booted into the cluster.

Enter a value for ORCHID's ALLOCLASS parameter [7]:
Does this cluster contain a quorum disk [N]? [Return]

The EXPECTED_VOTES system parameter of members of a cluster indicates the total
number of votes present when all cluster members are booted, and is used to determine
the minimum number of votes (QUORUM) needed for cluster operation.

EXPECTED_VOTES value for this cluster: 1

Warning:  Setting EXPECTED_VOTES to 1 allows this node to boot without
being able to see any other nodes in the cluster.  If there is
another instance of the cluster in existence that is unreachable via SCS but shares
common drives (such as a Fibrechannel fabric)this may result in severe disk corruption.

Do you wish to re-enter the value of EXPECTED_VOTES [Y]? N

The use of a quorum disk is recommended for small clusters to maintain cluster
quorum if cluster availability with only a single cluster node is a requirement.

For complete instructions, check the section on configuring a cluster in
the "OpenVMS Cluster Systems" manual.


  WARNING: ORCHID will be a voting cluster member. EXPECTED_VOTES for
           this and every other cluster member should be adjusted at
           a convenient time before a reboot. For complete instructions,
           check the section on configuring a cluster in the "OpenVMS
           Cluster Systems" manual.

Execute AUTOGEN to compute the SYSGEN parameters for your configuration and reboot
ORCHID with the new parameters. This is necessary before ORCHID can become a cluster member.

Do you want to run AUTOGEN now [Y]? N

    Please run AUTOGEN to reboot ORCHID:

     ORCHID$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT 18

1

Node ORCHID is currently a standalone, Integrity server node and is made as a member of a cluster. Only LAN or IP is used for cluster communication and no other interconnect is supported.

2

Select IP for cluster communication in addition to LAN by entering "YES". The SYSGEN parameter, NISCS_USE_UDP is set to 1 and PEDRIVER uses IP in addition to LAN for cluster communication when the node is rebooted.

3

The UDP port number to be used for cluster communication. The UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number and this port number must not be used by any other application.

4

You can enable IP multicast for cluster communication if your environment allows IP multicast traffic between cluster nodes. Check with your network administrator to see if IP multicasting is enabled in your environment.

5

Enter the IP multicast address for cluster, if IP multicasting is enabled. By default, the IP multicast address is selected from the administratively scoped IP multicast address range of 239.242.x.y. The last two octets x and y are generated based on the cluster group number. In the above example cluster group number is 1985 and is calculates as follows:
X= 1985/256
Y= 1985 - (256 *x)
The system administrator can override the default multicast address by a unique address for their environment.

6

TTL is the time to live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets.

7

Enter "Yes" to enter the IP unicast address for remote nodes of the cluster which are not reachable using IP multicast address.

8

In this example, 10.0.2.2 is the IP unicast address of node TULIP. Although, IP multicast is selected, TULIP's IP address is entered because the IP multicast connectivity between SITE A and SITE B is presumed to be non-existent in this example. NOTE: Enter the list of IP unicast address of the cluster. All the information entered in [2], [3], [5], [6], and [7] are entered in the SYS$SYSTEM:PE$IP_CONFIG.DAT file. The PE$IP_CONFIG.DAT file is generated as shown in following example. Also, the Unicast list in PE$IP_CONFIG.DAT in the local node should contain the remote node IP address for the local node to allow the remote node to join the cluster.

In this example, ORCHID must have TULIP's IP address and TULIP must have ORCHID's IP address. SYSTEM:PE$IP_CONFIG.DAT in node ORCHID:
! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL-

2008 14:14:06.57

multicast_address=239.242.7.193

ttl=32

udp_port=49152

unicast=10.0.2.2

9

Press Return after entering the unicast list.

10

CLUSTER_CONFIG_LAN.COM invokes TCPIP$CONFIG.COM to configure the IP interfaces used for cluster communication. Currently, ORCHID is a standalone node, when TCPIP$CONFIG is invoked by the CLUSTER_CONFIG_LAN procedure, TCP/IP configuration is limited to IPCI. The interface, IE0 is selected for enabling cluster communications. Note: TCPIP$CONFIG must be invoked after joining the cluster for other TCP/IP configuration, such as, FTP, TELNET.

11

IPv4 address for the IE0 interface is 10.0.1.2

12

Network mask for the IE0 interface is 255.255.254.0

13

The IE0 interface information along with network mask is entered in the TCPIP$CLUSTER.DAT file.

14

Exit the interface menu after selecting the interface for cluster communication.

15

The default gateway address for the interface IE0 is entered. Only one default gateway address is allowed for Cluster over IP communication.

16

After the interface and default gateway are selected, TCPIP$CONFIG updates the TCPIP$CLUSTER.DAT with the default route or gateway information. This also completes the TCPIP$CONFIG required for cluster communications using IP. The interface information along with the default route is entered in the TCPIP$CLUSTER.DAT file as shown:
interface=IE0,EIA0,10.0.1.2,255.255.254.0

default_route=10.0.1.1

17

Proceed with cluster configuration.

18

After rebooting the system, run AUTOGEN. PEDRIVER in ORCHID will start using IP in addition to LAN for cluster communication and must be able to join TULIP.

8.2.3.2. Adding a new Node to a Cluster over IP

This section describes how to add a new node, JASMIN to an existing two-node cluster. Nodes, ORCHID and TULIP are currently members of a two-node cluster, which are at SITE A and SITE B. For more information about configuring a node with IP as interconnect, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”. Node JASMIN is currently a standalone node at SITE C with IP connectivity to both SITE A and SITE B.

Step 1. Ensuring IP Connectivity

Ensure that the IP connectivity between the node JASMIN and the nodes ORCHID and TULIP is working fine. Use the TCP/IP PING utility to test the IP connectivity between JASMIN and other nodes, ORCHID and TULIP.

If PING fails, set up TCP/IP configuration properly so that node JASMIN can ping both ORCHID and TULIP.

Step 2. Executing the CLUSTER_CONFIG_LAN.COM

Execute CLUSTER_CONFIG_LAN.COM on node JASMIN. Because, the node JASMIN is a standalone node, complete the procedure described in Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”.Complete the sequence of steps provided in the following example while entering the unicast list.
Do you want to enter unicast address(es)(Y/N)[Y]?[Return]
What is the unicast address[Press [RETURN] to end the list]? 10.0.3.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2 1
What is the unicast address[Press [RETURN] to end the list]? [Return]
SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN shown below
! CLUSTER_CONFIG_LAN creating for CHANGE operation on 10-JUL-2008 14:14:06.57
multicast_address=239.242.7.193
ttl=32
udp_port=49152
unicast=10.0.3.2
Unicast=10.0.2.2
Unicast=10.0.1.2 

1

Enter the IP address of JASMIN, ORCHID and TULIP while configuring the node JASMIN.

Note

The unicast list must be consistent in all nodes of the cluster. Hence, while entering the unicast list in JASMIN, enter the IP addresses of all the three nodes of the cluster (that is, JASMIN, ORCHID and TULIP). You can also enter the local nodes IP addresses along with the Unicast list as it facilitates system management.

Step 3. Completing the Configuration Procedure Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP” for more details.

Step 4. Updating the PE$IP_CONFIG.DAT File To ensure that the nodes join the cluster, it is required to have PE$IP_CONFIG.DAT consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes, ORCHID, and TULIP.

Step 5. Refreshing the Unicast List

On both ORCHID and TULIP, to update the new unicast list in the PE$IP_CONFIG.DAT file, enter the following command for PEDRIVER:
$MC SCACP RELOAD
You can also use SYSMAN and run the command cluster wide.

Note

The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members of the SYS$SYSTEM:PE$IP_CONFIG.DAT file.

Step 6. Running AUTOGEN and Rebooting the Node

After the first boot of JASMIN, AUTOGEN.COM runs automatically to join the existing cluster consisting of nodes ORCHID and LOTUS.
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

8.2.3.3. Adding a new Node to a Cluster over IP with a Shared System Disk

This section describes how to add a new node JASMIN that has a shared system disk of TULIP. ORCHID and TULIP are currently members of two-node cluster which are at SITE A and SITE B.

Step 1. Obtaining the Interface Information

Node JASMIN is an OpenVMS Alpha node and is directly connected to the system disk of one of the node TULIP. In this configuration, Node JASMIN is connected in network, but not yet booted.

To configure a Cluster over IP, the interface information of JASMIN is required. This information can be obtained from the ">>>" prompt on JASMIN by executing the following command:
P00>>>SHOW DEVICE
dga5245.1003.0.3.0         $1$DGA5245   COMPAQ HSV110 (C)COMPAQ  3028
dga5245.1004.0.3.0         $1$DGA5245   COMPAQ HSV110 (C)COMPAQ  3028
dga5890.1001.0.3.0         $1$DGA5890   COMPAQ HSV110 (C)COMPAQ  3028
dga5890.1002.0.3.0         $1$DGA5890   COMPAQ HSV110 (C)COMPAQ  3028
dka0.0.0.2004.0            DKA0              COMPAQ BD03685A24  HPB7
dka100.1.0.2004.0          DKA100            COMPAQ BD01864552  3B08
dka200.2.0.2004.0          DKA200            COMPAQ BD00911934  3B00
dqa0.0.0.15.0              DQA0       HL-DT-ST CD-ROM GCR-8480  2.11
dva0.0.0.1000.0            DVA0
eia0.0.0.2005.0            EIA0              00-06-2B-03-2D-7D
pga0.0.0.3.0               PGA0        WWN 1000-0000-c92a-78e9
pka0.7.0.2004.0            PKA0                  SCSI Bus ID 7
pkb0.6.0.2.0               PKB0                  SCSI Bus ID 6  5.57
P00>>>

From the output, the interface will be EIA0 on which the IP address will be configured and can be used for cluster formation.

To obtain the interface information on Integrity server system, execute the following commands on the EFI Shell:
shell>fs0:
fs0:\> cd efi
fs0:\EFI> cd vms
fs0:\EFI\VMS> vms_show device
VMS: EIA0               00-30-6E-F3-EC-6E
EFI: Acpi(HWP0002,0)/Pci(3|0)/Mac(00306EF3EC6E)
VMS: DKA100             HP 36.4GST336754LC      HPC2
EFI: Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun1,Lun0)
VMS: DKA0               HP 36.4GMAS3367NC       HPC3    X8_3_XBJL
EFI: fs0: Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)
VMS: EWA0               00-30-6E-F3-3C-28
EFI: Acpi(HWP0002,100)/Pci(2|0)/Mac(00306EF33C28)
fs0:\EFI\VMS> 

From the output, the interface will be EIA0. Here fs0: is the partition of the shared system disk.

Step 2. Executing CLUSTER_CONFIG_LAN.COM

Execute the following command procedure on node TULIP:
TULIP$ @SYS$SYSROOT:[SYSMGR]CLUSTER_CONFIG_LAN.COM;1

                   Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an Alpha System

DECnet Phase IV is installed on this node.
Alpha satellites will use LANCP, not DECnet, for MOP downline loading.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    TULIP is an Alpha system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an Alpha node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for Tulip.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 1

    This ADD function will add a new Alpha node to the cluster.

  WARNING: If the node being added is a voting member, EXPECTED_VOTES for
           every cluster member must be adjusted. For complete
           instructions check the section on configuring a cluster in the
           "OpenVMS Cluster Systems" manual.

  CAUTION: If this cluster is running with multiple system disks and
           common system files will be used, please, do not proceed
           unless appropriate logical names are defined for cluster
           common files in SYLOGICALS.COM. For instructions, refer to
           the "OpenVMS Cluster Systems" manual.

Do you want to continue [Y]?Y
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? Y
Will the node be a satellite [Y]? N
What is the node's SCS node name? JASMIN
What is the node's SCSSYSTEMID number? 14487
Will JASMIN be a boot server [Y]?Y

This procedure will now ask you for the device name of JASMIN's system root.
The default device name (DISK$TULIPSYS:) is the logical volume name of
    SYS$SYSDEVICE:.

What is the device name for JASMIN's system root
[default DISK$TULIPSYS:]?
What is the name of JASMIN's system root [SYS3]?SYS3
    Creating directory tree SYS3 ...
    System root SYS3 created
ENABLE IP for cluster communications (Y/N)? Y
UDP port number to be used for Cluster Communication over IP[49152]?[Return]
Enable IP multicast for cluster communication(Y/N)[Y]?Y
What is the IP multicast address[224.0.0.3]? [Return]
What is the TTL (time to live) value for IP multicast packets [1] ?  [Return]
Do you want to enter unicast address(es)(Y/N)[Y]?Y
What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3
What is the unicast address[Press [RETURN] to end the list]?  [Return]

   *****************************************************************
        Cluster Communications  over IP  has  been  enabled.  Now
        CLUSTER_CONFIG_LAN will run the  SYS$MANAGER:TCPIP$CONFIG
        procedure. Please select the IP interfaces to be used for
        Cluster Communications  over IP  (IPCI). This can be done
        selecting "Core Environment"  option from the main menu
        followed by the "Interfaces"  option.  You may also use
        this opportunity to configure other aspects.
   ****************************************************************

Press Return to continue ...

       Checking TCP/IP Services for OpenVMS configuration database files.

        TCP/IP Services for OpenVMS Configuration Menu

        Configuration options:

                 1  -  Core environment
                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option: 1


       TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option: 2


    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=TULIP, Active=TULIP

 Configuration options:

                 0  -  Set The Target Node (Current Node: TULIP)
                 1  -  WE0 Menu (EWA0: TwistedPair 1000mbps)
                 2  -  10.0.2.2/8    Tulip                 Configured,IPCI
                 3  -  WE1 Menu (EWB0: TwistedPair 1000mbps)
                 4  -  WE2 Menu (EWC0: TwistedPair 1000mbps)
                 5  -  WE3 Menu (EWD0: TwistedPair 1000mbps)
                 6  -  WE4 Menu (EWE0: TwistedPair 1000mbps)
                 7  -  WE5 Menu (EWF0: TwistedPair 1000mbps)
                 8  -  WE6 Menu (EWG0: Multimode 10000mbps)
                 9  -  WE7 Menu (EWH0: TwistedPair 1000mbps)
                10  -  IE0 Menu (EIA0: TwistedPair 100mbps)
                11  -  IE1 Menu (EIB0: TwistedPair 100mbps)

Enter configuration option or press ENTER key to continue: 0 1
Enter name of node to manage [TULIP]: JASMIN
JASMIN is not currently a cluster member.
* Continue configuring JASMIN [NO]: Y 2
Enter system device for JASMIN [$10$DGA165:]:3
Enter system root for JASMIN []: SYS3 4
    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=Not Configured

 Configuration options:

                 0  -  Set The Target Node (Current Node: JASMIN - $10$DGA165:[sys3.])
                 A  -  Add an Interface
                [E] -  Exit menu

Enter configuration option: A
Enter controller name (e.g. EIA or EWC, etc): [ENTER when done] EIA 5

    Controller Name       :  EIA
    TCP/IP Interface Name :  IE0

* Is this correct [NO]: Y
Interface Menu: IE0


      TCP/IP Services for OpenVMS Interface IE0 Configuration Menu (Node: JASMIN)

 Configuration options:

                 1  - Add a primary address on IE0
                 2  - Add an alias address on IE0
                 3  - Enable DHCP client to manage address on IE0
                [E] - Exit menu

Enter configuration option: 1 6
* Is this address used by Clusters over IP (IPCI) [NO]: Y 7
    IPv4 Address may be entered with CIDR bits suffix.
    E.g. For a 16-bit netmask enter 10.0.1.1/16

Enter IPv4 Address []: 10.0.2.3
Default netmask calculated from class of IP address: 255.0.0.0

    IPv4 Netmask may be entered in dotted decimal notation,
    (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16)

Enter Netmask or CIDR bits [255.0.0.0]:
Enter hostname []: JASMIN

Requested configuration:

      Node     : JASMIN
      Interface: IE0
      IPCI     : Yes
      Address  : 10.0.2.3/8
      Netmask  : 255.0.0.0 (CIDR bits: 8)
      Hostname : JASMIN

* Is this correct [YES]:Y
Added hostname JASMIN (10.0.2.3) to host database

NOTE:
  The system hostname is not configured.
  It will now be set to jasmin (10.0.2.3).
  This can be changed later via the Interface Configuration Menu.

Updated system hostname in configuration database

Added address IE0:10.0.2.3 to configuration database
Updated Interface in IPCI configuration file: $10$DGA165:[SYS3.SYSEXE]TCPIP$CLUSTER.DAT;

Updated Default Route in IPCI configuration file: $10$DGA165:[SYS3.SYSEXE]TCPIP$CLUSTER.DAT;


    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=JASMIN

 Configuration options:
                 0  -  Set The Target Node (Current Node: JASMIN - $10$DGA165:[sys3.]
                 1  -  IE0 Menu (EIA0:)
                 2  -  10.0.2.3/8          JASMIN                Configured,IPCI
                 I  -  Information about your configuration
                 A  -  Add an Interface
                [E] -  Exit menu

Enter configuration option:


       TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option:


        TCP/IP Services for OpenVMS Configuration Menu

        Configuration options:

                 1  -  Core environment
                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option:

SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN's root shown below

! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05
multicast_address=224.0.0.3
ttl=1
udp_port=49152
unicast=10.0.2.3
unicast=10.0.2.2
unicast=10.0.1.2

SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node JASMIN's root shown below

interface=IE0,EIA0,10.0.2.3,255.0.0.0
default_route=16.116.40.1

1

In the TCP/IP configuration, select option 0 to set the target node to JASMIN, which is the new node added to the cluster.


2

Proceed with the configuration procedure to configure node JASMIN.


3

Enter the system device for JASMIN, which is $10$DGA165.


4

Enter JASMIN's root, which is SYS3.


5

Enter the controller information on which IP will be configured for cluster traffic. This is the controller information that has been obtained from the console of the machine JASMIN as explained in the beginning of the configuration.


6

Select the option to add the primary address for IE0 (IP interface name of controller EIA).


7

Enable the use of IE0 for Cluster over IP and proceed with the rest of the configuration.

Step 3. Completing the Configuration Procedure

Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure. For more information, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”.

Step 4. Updating the PE$IP_CONFIG.DAT file

To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes, ORCHID and TULIP.

Step 5. Refreshing the Unicast list

On both ORCHID and TULIP, to update the new Unicast list in the PE$IP_CONFIG.DAT file, enter the following command for PEDRIVER:
$ MC SCACP RELOAD
You can also use SYSMAN and run the command cluster wide.

Note

The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members SYS$SYSTEM:PE$IP_CONFIG.DAT file.

Step 6. Running AUTOGEN and Rebooting the Node

After the first boot of JASMIN, AUTOGEN.COM runs automatically. JASMIN will now be able to join the existing cluster consisting of nodes ORCHID and LOTUS.
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

8.2.3.4. Adding an Integrity server Satellite node to a Cluster over IP

This section describes how to add a satellite node to an existing two-node cluster. JASMIN is an Integrity server satellite node and is added to a cluster that has two nodes, ORCHID and TULIP. TULIP is the boot server for the satellite node.

Note

For both Alpha and Integrity server satellite nodes, the satellite node and its boot server must exist in the same LAN segment.

Step 1. Selecting the Interface for Satellite Booting

To select the interface to be used for satellite booting, assume that the satellite node does not have any disk running OpenVMS connected to it. Note: If you are adding Alpha systems as satellite nodes, you can get the information from ">>>" prompt by executing the following command:
P00>>>SHOW DEVICE
dga5245.1003.0.3.0         $1$DGA5245   COMPAQ HSV110 (C)COMPAQ  3028
dga5245.1004.0.3.0         $1$DGA5245   COMPAQ HSV110 (C)COMPAQ  3028
dga5890.1001.0.3.0         $1$DGA5890   COMPAQ HSV110 (C)COMPAQ  3028
dga5890.1002.0.3.0         $1$DGA5890   COMPAQ HSV110 (C)COMPAQ  3028
dka0.0.0.2004.0            DKA0              COMPAQ BD03685A24  HPB7
dka100.1.0.2004.0          DKA100            COMPAQ BD01864552  3B08
dka200.2.0.2004.0          DKA200            COMPAQ BD00911934  3B00
dqa0.0.0.15.0              DQA0       HL-DT-ST CD-ROM GCR-8480  2.11
dva0.0.0.1000.0            DVA0
eia0.0.0.2005.0            EIA0              00-06-2B-03-2D-7D
pga0.0.0.3.0               PGA0        WWN 1000-0000-c92a-78e9
pka0.7.0.2004.0            PKA0                  SCSI Bus ID 7
pkb0.6.0.2.0               PKB0                  SCSI Bus ID 6  5.57
P00>>> 
From the output, the LAN interface will be EIA0 on which IP address is configured and used for cluster configuration.

Note

The Alpha console uses the MOP protocol for network load of satellite systems. Because the MOP protocol is non-routable, the satellite boot server or servers and all satellites booting from them must reside in the same LAN. In addition, the boot server must have at least one LAN device enabled for cluster communications to permit the Alpha satellite nodes to access the system disk.

On Integrity server systems, the interface name either starts with 'EI' or 'EW'. If it is the first interface, it will be EIA0 or EWA0. Note the mac address of the interface that you want to use from the Shell prompt. To obtain the interface information on Integrity servers, execute the following command on EFI Shell:
Shell> lanaddress

LAN Address Information

   LAN Address        Path
   -----------------  ----------------------------------------
   Mac(00306E4A133F)  Acpi(HWP0002,0)/Pci(3|0)/Mac(00306E4A133F))
  *Mac(00306E4A02F9)  Acpi(HWP0002,100)/Pci(2|0)/Mac(00306E4A02F9))

Shell>

Assuming that the interface which is active is EIA0, configure the satellite with EIA0, if it does not boot with EIA0 then try with EWA0 subsequently.

Step 2. Executing CLUSTER_CONFIG_LAN.COM

Execute CLUSTER_CONFIG_LAN on the boot server node TULIP and select the appropriate option as described in the following example:
TULIP$ @SYS$SYSROOT:[SYSMGR]CLUSTER_CONFIG_LAN.COM
                   Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading

    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    TULIP is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for TULIP.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]:

    This ADD function will add a new IA64 node to the cluster.

  WARNING: If the node being added is a voting member, EXPECTED_VOTES for
           every cluster member must be adjusted.  For complete instructions
           check the section on configuring a cluster in the "OpenVMS Cluster
           Systems" manual.

  CAUTION: If this cluster is running with multiple system disks and
           common system files will be used, please, do not proceed
           unless appropriate logical names are defined for cluster
           common files in SYLOGICALS.COM. For instructions, refer to
           the "OpenVMS Cluster Systems" manual.

Do you want to continue [Y]? Y
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N

Will the node be a satellite [Y]? [Return]
What is the node's SCS node name? JASMIN
What is the node's SCSSYSTEMID number? 25482

  WARNING:
    DECnet is not running.
    No DECnet databases will be updated with information on JASMIN.

Does JASMIN need to be registered in the DECnet namespace [N]?[Return]
What is the Cluster Alias fullname?

   This procedure will now ask you for the device name of JASMIN's system root.
    The default device name (DISK$TULIPSYS:) is the logical volume name of
    SYS$SYSDEVICE:.

What is the device name for JASMIN's system root [default DISK$TULIPSYS:]? [Return]
What is the name of JASMIN's system root [SYS14]? [Return]
What is JASMIN's LAN adapter hardware address? 00-30-6E-4A-02-F9 1
What is JASMIN's TCP/IP address [10.0.2.3]? [Return] 2
What is JASMIN's TCP/IP gateway or gateways (leave blank if none)? 10.0.2.1 3
What is JASMIN's TCP/IP network mask [255.255.255.0]? 255.255.254.0 4

  NOTE:  Make sure to set the VMS_FLAGS console variable
         to 0,200000 on node JASMIN so it will use
         the memory-disk method to boot as a satellite.
         The command to update this variable from the
         console EFI shell of JASMIN is:
           set vms_flags "0,200000"

Allow conversational bootstraps on JASMIN [N]? [Return]

    The following workstation windowing options are available:

       1. No workstation software
       2. DECwindows Workstation Software

Enter choice [1]:
    Creating directory tree SYS14 ...
    System root SYS14 created
ENABLE IP for cluster communications (Y/N)? Y 5
UDP port number to be used for Cluster Communication over IP[49152]? [Return] 6
Enable IP multicast for cluster communication(Y/N)[Y]? Y 7
What is IP the multicast address[224.0.0.3]? [Return]
What is the TTL (time to live) value for IP multicast packets [1] ? 32]? [Return] 8
Do you want to enter unicast address(es)(Y/N)[Y]? Y 9
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2
What is the unicast address[Press [RETURN] to end the list]? [Return]

   *****************************************************************
        Cluster Communications  over IP  has  been  enabled.  Now
        CLUSTER_CONFIG_LAN will run the  SYS$MANAGER:TCPIP$CONFIG
        procedure. Please select the IP interfaces to be used for
        Cluster Communications  over IP  (IPCI). This can be done
        selecting "Core Environment"  option from the main menu
        followed by the "Interfaces"  option.  You may also use
        this opportunity to configure other aspects.
   ****************************************************************

Press Return to continue ...

        Checking TCP/IP Services for OpenVMS configuration database files.

        Configuration options:

                 1  -  Core environment
                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option: 1


        TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option: 2

TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=[], Active=[]

 Configuration options:

   0  -  Set The Target Node (Current Node: TULIP)
   1  -  IE0 Menu (EIA0: TwistedPair 100mbps)
   2  -  15.146.235.222/23   *noname*              Configured
   3  -  15.146.235.254/23   []                    IPCI
   4  -  IE1 Menu (EIB0: TwistedPair 100mbps)
   5  -  15.146.235.222/23   *noname*              Configured,Active
   I  -  Information about your configuration
  [E] -  Exit menu

Enter configuration option: 0 10
Enter name of node to manage [TULIP]: JASMIN
JASMIN is not currently a cluster member.
* Continue configuring JASMIN [NO]: YES 11
Enter system device for JASMIN [DSA2:]: 12
Enter system root for JASMIN []: SYS14 13
      TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=JASMIN

 Configuration options:

   0  -  Set The Target Node (Current Node: JASMIN - DSA2:[SYS14]
   A  -  Add an Interface
  [E] -  Exit menu

Enter configuration option: a
Enter controller name (e.g. EIA or EWC, etc): [ENTER when done] EIA 14
    Controller Name       :  EIA
    TCP/IP Interface Name :  IE0

* Is this correct [NO]: y
Interface Menu:IE0


      TCP/IP Services for OpenVMS Interface IE0 Configuration Menu (Node: JASMIN)

 Configuration options:

         1  - Add a primary address on IE0
         2  - Add an alias address on IE0
         3  - Enable DHCP client to manage address on IE0
        [E] - Exit menu

Enter configuration option: 1 15
* Is this address used by Clusters over IP (IPCI) [NO]: Y 16
    IPv4 Address may be entered with CIDR bits suffix.
    E.g. For a 16-bit netmask enter 10.0.1.1/16

Enter IPv4 Address []: 10.0.2.3
Default netmask calculated from class of IP address: 255.0.0.0

    IPv4 Netmask may be entered in dotted decimal notation,
    (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16)

Enter Netmask or CIDR bits [255.0.0.0]:
Enter hostname []: JASMIN

Requested configuration:

      Node     : JASMIN
      Interface: IE0
      IPCI     : Yes
      Address  : 10.0.2.3/8
      Netmask  : 255.0.0.0 (CIDR bits: 8)
      Hostname : jasmin

* Is this correct [YES]:
Added hostname jasmin (10.0.2.3) to host database

NOTE:
  The system hostname is not configured.
  It will now be set to jasmin (10.0.2.3).
  This can be changed later via the Interface Configuration Menu.

Updated system hostname in configuration database

Added address IE1:10.0.2.3 to configuration database
Updated Interface in IPCI configuration file: DSA2:[SYS14.SYSEXE]TCPIP$CLUSTER.DAT;

Updated Default Route in IPCI configuration file: DSA2:[SYS14.SYSEXE]TCPIP$CLUSTER.DAT;


      TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=JASMIN

 Configuration options:

   0  -  Set The Target Node (Current Node: JASMIN - DSA2:[SYS14.])
   1  -  IE1 Menu (EIB0:)
   2  -  10.0.2.3/8     JASMIN                  Configured,IPCI
   I  -  Information about your configuration
   A  -  Add an Interface
  [E] -  Exit menu

Enter configuration option:

        TCP/IP Services for OpenVMS Core Environment Configuration Menu

        Configuration options:

                 1  -  Domain
                 2  -  Interfaces
                 3  -  Routing
                 4  -  BIND Resolver
                 5  -  Time Zone
                 A  -  Configure options 1 - 5
                [E] -  Exit menu

Enter configuration option: E


        TCP/IP Services for OpenVMS Configuration Menu

        Configuration options:

                 1  -  Core environment
                 2  -  Client components
                 3  -  Server components
                 4  -  Optional components
                 5  -  Shutdown TCP/IP Services for OpenVMS
                 6  -  Startup TCP/IP Services for OpenVMS
                 7  -  Run tests
                 A  -  Configure options 1 - 4
                [E] -  Exit configuration procedure

Enter configuration option: E

SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node JASMIN's root shown below

! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05
multicast_address=224.0.0.3
ttl=32
udp_port=49152
unicast=10.0.2.3
unicast=10.0.2.2
unicast=10.0.1.2

SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node JASMIN's root shown below

interface=IE0,EIA0,10.0.2.3,255.0.0.0
default_route=16.116.40.1 

Note

Assuming that the interface which is active is EIA, configure the satellite with EIA, if it does not boot with EIA then try with EIB subsequently. If the wrong interface name is given, satellite node fails with the messages during booting.


1

Enter the LAN adapter's hardware address.


2

Enter the TCP/IP address.


3

Enter the TCP/IP gateway.


4

Enter the TCP/IP network mask address.


5

Enable IP for cluster communication.


6

The UDP port number to be used for cluster communication. The UDP port number must be same on all members of the cluster. Also, ensure that there is no other cluster in your environment using the same UDP port number and this port number must not be used by any other application.


7

Enter the IP multicast address for cluster, if IP multicasting is enabled. By default, the IP multicast address is selected from the administratively scoped IP multicast address range of 239.242.x.y. The last two octets x and y are generated based on the cluster group number. In the above example, the cluster group number is 1985 and can be calculated as follows:
X= 1985/256
Y= 1985 - (256 *x)

The system administrator can override the default multicast address with a unique address for their environment. The multicast address is modified based on the cluster group number or it can be added to .DAT file.


8

TTL is the time-to-live for IP multicast packets. It specifies the number of hops allowed for IP multicast packets.


9

Enter "yes" to enter the IP Unicast address of remote nodes of the cluster, which are not reachable using IP multicast address.


10

In the TCP/IP configuration, select option 0 to set the target node to JASMIN, which is the satellite node, and will be added to the cluster.


11

Proceed with configuration steps to configure node JASMIN.


12

Enter the system device for JASMIN, which DSA2.


13

Enter JASMIN's root, which SYS14.


14

Enter the controller information on which IP will be configured for cluster communication. The controller information is obtained from the console of JASMIN as explained in the beginning of the configuration.


15

Select an option to add a primary address for IE0 (IP interface name of controller EIA).


16

Enable the use of IE0 for Cluster over IP and proceed with the rest of the configuration.

Step 3. Executing the CLUSTER_CONFIG_LAN.COM Procedure

Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure.
Adjusting protection on DSA2:[SYS14.][SYSEXE]PE$IP_CONFIG.DAT;1
Will JASMIN be a disk server [N]? Y
Enter a value for JASMIN's ALLOCLASS parameter [0]: 15
Updating BOOTP database with satellite information for JASMIN..
Size of pagefile for JASMIN [RETURN for AUTOGEN sizing]?

A temporary pagefile will be created until resizing by AUTOGEN. The
default size below is arbitrary and may or may not be appropriate.

Size of temporary pagefile [10000]? [Return]
Size of swap file for JASMIN [RETURN for AUTOGEN sizing]? [Return]

A temporary swap file will be created until resizing by AUTOGEN. The
default size below is arbitrary and may or may not be appropriate.

Size of temporary swap file [8000]? [Return]

   NOTE:  IA64 satellite node JASMIN requires DOSD if capturing the
          system state in a dumpfile is desired after a system crash.

Will a local disk on JASMIN be used for paging and swapping (Y/N)? N

    If you specify a device other than DISK$TULIPSYS: for JASMIN's
    page and swap files, this procedure will create PAGEFILE_JASMIN.SYS
    and SWAPFILE_JASMIN.SYS in the <SYSEXE> directory on the device you
    specify.

What is the device name for the page and swap files [DISK$TULIPSYS:]? [Return]
%SYSGEN-I-CREATED, DSA2:<SYS14.SYSEXE>PAGEFILE.SYS;1 created
%SYSGEN-I-CREATED, DSA2:<SYS14.SYSEXE>SWAPFILE.SYS;1 created
    The configuration procedure has completed successfully. 

The node JASMIN is configured to join the cluster. After the first boot of JASMIN, AUTOGEN.COM will run automatically.

Step 4. Updating the PE$IP_CONFIG.DAT File

To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN's root to the other nodes, ORCHID and TULIP.

Step 5. Refreshing the Unicast list

On both ORCHID and TULIP, to update the new unicast list in the PE$IP_CONFIG.DAT file, enter the following command for PEDRIVER:
$MC SCACP RELOAD
You can also use SYSMAN and run the command cluster wide.

Note

The following rule is applicable when IP unicast address is used for node discovery. A node is allowed to join the cluster only if its IP address is present in the existing members of the SYS$SYSTEM:PE$IP_CONFIG.DAT file.

Step 6. Running AUTOGEN and Rebooting the Node

After the first boot of JASMIN, AUTOGEN.COM runs automatically. JASMIN will now be able to join the existing cluster consisting of nodes ORCHID and TULIP.
JASMIN$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

8.2.3.5. Adding an Integrity server Node to a Cluster over IP with Logical LAN Failover set

This section describes how to add a node, ORCHID to an existing two-node cluster, JASMIN and TULIP. The Logical LAN failover set is created and configured on ORCHID. ORCHID can survive failure if a local LAN card fails and it will switchover to other interface configured in the logical LAN failover set.

Step 1. Configuring the Logical LAN Failover set

Execute the following commands to create a logical LAN failover set.
$ MC LANCP
LANCP>DEFINE DEVICE LLB/ENABLE/FAILOVER=(EIA0, EIB0))
Reboot the system and during reboot, the following console message is displayed:
%LLB0, Logical LAN event at  2-SEP-2008 14:52:50.06
%LLB0, Logical LAN failset device created

Step 2: Executing CLUSTER_CONFIG_LAN

Execute CLUSTER_CONFIG_LAN.COM on node ORCHID and select the appropriate option as shown:
ORCHID$@SYS$MANAGER:CLUSTER_CONFIG_LAN
                 Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    This IA64 node is not currently a cluster member.

MAIN Menu

   1. ADD ORCHID to existing cluster, or form a new cluster.
   2. MAKE a directory structure for a new root on a system disk.
   3. DELETE a root from a system disk.
   4. EXIT from this procedure.

Enter choice [4]: 1
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? n
What is the node's SCS node name? orchid
    IA64 node, using LAN/IP for cluster communications.  PEDRIVER will be loaded.
   No other cluster interconnects are supported for IA64 nodes.
Enter this cluster's group number: 1985
Enter this cluster's password:
Re-enter this cluster's password for verification:

ENABLE IP for cluster communications (Y/N)? Y
UDP port number to be used for Cluster Communication over IP[49152]? [Return]
Enable IP multicast for cluster communication(Y/N)[Y]? [Return]
What is IP the multicast address[239.242.7.193]? 239.242.7.193
What is the TTL (time to live) value for IP multicast packets [32]? [Return]
Do you want to enter unicast address(es)(Y/N)[Y]? [Return]
What is the unicast address[Press [RETURN] to end the list]? 10.0.1.2
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.3
What is the unicast address[Press [RETURN] to end the list]? 10.0.2.2
What is the unicast address[Press [RETURN] to end the list]? [Return]

   *****************************************************************
        Cluster Communications  over IP  has  been  enabled.  Now
        CLUSTER_CONFIG_LAN will run the  SYS$MANAGER:TCPIP$CONFIG
        procedure. Please select the IP interfaces to be used for
        Cluster Communications  over IP  (IPCI). This can be done
        selecting "Core Environment"  option from the main menu
        followed by the "Interfaces"  option.  You may also use
        this opportunity to configure other aspects.
   ****************************************************************
Press Return to continue ...

                TCP/IP Network Configuration Procedure

        This procedure helps you define the parameters required
        to run TCP/IP Services for OpenVMS on this system.

%TCPIP-I-IPCI, TCP/IP Configuration is limited to IPCI.
-TCPIP-I-IPCI, Rerun TCPIP$CONFIG after joining the cluster.



    TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=Not Configured, Active=nodeg

 Configuration options:

   0  -  Set The Target Node (Current Node: ORCHID)
   1  -  LE0 Menu (LLA0: TwistedPair 100mbps)
   2  -  IE1 Menu (EIB0: TwistedPair 100mbps)
  [E] -  Exit menu

Enter configuration option: 1

* IPCI Address Configuration *

Only IPCI addresses can be configured in the current environment.
After configuring your IPCI address(es) it will be necessary to
run TCPIP$CONFIG once your node has joined the cluster.


    IPv4 Address may be entered with CIDR bits suffix.
    E.g. For a 16-bit netmask enter 10.0.1.1/16

Enter IPv4 Address []:10.0.1.2
Default netmask calculated from class of IP address: 255.0.0.0

    IPv4 Netmask may be entered in dotted decimal notation,
    (e.g. 255.255.0.0), or as number of CIDR bits (e.g. 16)

Enter Netmask or CIDR bits [255.0.0.0]: 255.255.255.0

Requested configuration:

      Node     : ORCHID
      Interface: IE0
      IPCI     : Yes
      Address  : 10.0.1.2/24
      Netmask  : 255.255.254.0 (CIDR bits: 23)

* Is this correct [YES]:
Updated Interface in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.
DAT;


   TCP/IP Services for OpenVMS Interface & Address Configuration Menu

 Hostname Details: Configured=Not Configured, Active=nodeg

 Configuration options:

 0  -  Set The Target Node (Current Node: ORCHID)
 1  -  LE0 Menu (LLA0: TwistedPair 100mbps)
 2  -  10.0.1.2/24  ORCHID  IPCI
 3  -  IE1 Menu (EIB0: TwistedPair 100mbps)
[E] -  Exit menu

Enter configuration option: E
Enter your Default Gateway address []: 10.0.1.1
* The default gateway will be: 10.0.1.1  Correct [NO]: YES
Updated Default Route in IPCI configuration file: SYS$SYSROOT:[SYSEXE]TCPIP$CLUSTER.DAT;
TCPIP-I-IPCIDONE, Finished configuring IPCI address information.


SYS$SYSTEM:PE$IP_CONFIG.DAT file generated in node ORCHID's root shown below

! CLUSTER_CONFIG_LAN creating for CHANGE operation on 15-JUL-2008 15:23:56.05
multicast_address=239.242.7.193
ttl=32
udp_port=49152
unicast=10.0.2.3
unicast=10.0.2.2
unicast=10.0.1.2

SYS$SYSTEM:TCPIP$CLUSTER.DAT file generated in node ORCHID's root shown below

interface=LE1,LLB0,10.0.1.2,255.0.0.0
default_route=10.0.1.1 

Step 3. Completing the Configuration Procedure

Continue to run the CLUSTER_CONFIG_LAN.COM to complete the cluster configuration procedure. For more information, see Section 8.2.3.1, “Creating and Configuring a Two-Node Cluster Using IP”.

Step 4. Updating the PE$IP_CONFIG.DAT file

To ensure that the nodes join the cluster, PE$IP_CONFIG.DAT must be consistent through all the members of the cluster. Copy the SYS$SYSTEM:PE$IP_CONFIG.DAT file that is created on node JASMIN to the other nodes ORCHID and TULIP.

Step 5. Refreshing the Unicast list

On both ORCHID and TULIP, to update the new unicast list in the PE$IP_CONFIG.DAT file, enter the following command for PEDRIVER:
$MC SCACP RELOAD

You can also use SYSMAN and run the command cluster wide.

Step 6. Running AUTOGEN and Rebooting the Node

After the first boot of ORCHID, AUTOGEN.COM will run automatically. ORCHID will now be able to join the existing cluster consisting of nodes JASMIN and LOTUS.
ORCHID$ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

8.2.4. Adding a Quorum Disk

To enable a quorum disk on a node or nodes, use the cluster configuration procedure as described in Table 8.5, “Preparing to Add a Quorum Disk Watcher”.
Table 8.5. Preparing to Add a Quorum Disk Watcher

IF...

THEN...

Other cluster nodes are already enabled as quorum disk watchers.

Perform the following steps:
  1. Log in to the computer that is to be enabled as the quorum disk watcher and run CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM.

  2. Execute the CHANGE function and select menu item 7 to enable a quorum disk. (See Section 8.4, “Changing Computer Characteristics”).

  3. Update the current system parameters and reboot the node. (See Section 8.6.1, “Updating Parameter Files”).

The cluster does not contain any quorum disk watchers.

Perform the following steps:
  1. Perform the preceding steps 1 and 2 for each node to be enabled as a quorum disk watcher.

  2. Reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”.

8.3. Removing Computers

To disable a computer as an OpenVMS Cluster member:
  1. Determine whether removing a member will cause you to lose quorum. Use the SHOW CLUSTER command to display the CL_QUORUM and CL_VOTES values.

    IF removing members...

    THEN...

    Will cause you to lose quorum

    Perform the steps in the following list:

    Caution: Do not perform these steps until you are ready to reboot the entire OpenVMS Cluster system. Because you are reducing quorum for the cluster, the votes cast by the node being removed could cause a cluster partition to be formed.

    Will not cause you to lose quorum

    Proceed as follows:
    • Perform an orderly shutdown on the node being removed by invoking the SYS$SYSTEM:SHUTDOWN.COM command procedure (described in Section 8.6.3, “Shutting Down a Single Node”).

    • If the node was a voting member, use the DCL command SET CLUSTER/EXPECTED_VOTES to reduce the value of quorum.

    Reference: Refer also to Section 10.11, “Restoring Cluster Quorum” for information about adjusting expected votes.

  2. Invoke CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM on an active OpenVMS Cluster computer and select the REMOVE option.

  3. Use the information in Table 8.6, “Preparing to Remove Computers from an OpenVMS Cluster ” to determine whether additional actions are required.
    Table 8.6. Preparing to Remove Computers from an OpenVMS Cluster

    IF...

    THEN...

    You are removing a voting member.

    You must, after the REMOVE function completes, reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”.

    The page and swap files for the computer being removed do not reside on the same disk as the computer's root directory tree.

    The REMOVE function does not delete these files. It displays a message warning that the files will not be deleted, as in Example 8.6, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Remove a Satellite with Local Page and Swap Files”. If you want to delete the files, you must do so after the REMOVE function completes.

    You are removing a computer from a cluster that uses DECdtm services.

    Make sure that you have followed the step-by-step instructions in the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual. These instructions describe how to remove a computer safely from the cluster, thereby preserving the integrity of your data.

Note: When the REMOVE function deletes the computer's entire root directory tree, it generates OpenVMS RMS informational messages while deleting the directory files. You can ignore these messages.

8.3.1. Example

Example 8.6, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Remove a Satellite with Local Page and Swap Files” illustrates the use of CLUSTER_CONFIG_LAN.COM on BHAGAT to remove satellite GOMTHI from the cluster.
Example 8.6. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Remove a Satellite with Local Page and Swap Files
$ @CLUSTER_CONFIG_LAN.COM
               Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 2

    The REMOVE command disables a node as a cluster member.

        o It deletes the node's root directory tree.
        o If the node has entries in SYS$DEVICES.DAT, any port allocation
          class for shared SCSI bus access on the node must be re-assigned.

    If the node being removed is a voting member, EXPECTED_VOTES
    in each remaining cluster member's MODPARAMS.DAT must be adjusted.
    The cluster must then be rebooted.

    For instructions, see the "OpenVMS Cluster Systems" manual.

    CAUTION: The REMOVE command does not remove the node name from any
    network databases. Also, if a satellite has been set up for booting
    with multiple hardware addresses, the satellite's aliases are not
    cleared from the LANACP boot database.

What is the node's SCS node name? GOMTHI
    Verifying BOOTP satellite node database...
    Verifying that $1$DKA0:[SYS10] is GOMTHI's root...
Are you sure you want to remove node GOMTHI (Y/N)? Y

  WARNING: GOMTHI's page and swap files will not be deleted.
           They do not reside on $1$DKA0:.


    Deleting directory tree $1$DKA0:<SYS10...>

%DELETE-I-FILDEL, $1$DKA0:<SYS10.SYS$I18N.LOCALES>SYSTEM.DIR;1 deleted (16 blocks)
.
.
.
.
%DELETE-I-FILDEL, $1$DKA0:<SYS10>VPM$SERVER.DIR;1 deleted (16 blocks)
%DELETE-I-TOTAL, 21 files deleted (336 blocks)
%DELETE-I-FILDEL, $1$DKA0:<0,0>SYS10.DIR;1 deleted (16 blocks)

    System root $1$DKA0:<SYS10> deleted.
    Updating BOOTP database...
    Removing rights identifier for GOMTHI...
    The configuration procedure has completed successfully. 

8.3.2. Removing a Quorum Disk

To disable a quorum disk on a node or nodes, use the cluster configuration command procedure as described in Table 8.7, “Preparing to Remove a Quorum Disk Watcher”.
Table 8.7. Preparing to Remove a Quorum Disk Watcher

IF...

THEN...

Other cluster nodes will still be enabled as quorum disk watchers.

Perform the following steps:
  1. Log in to the computer that is to be disabled as the quorum disk watcher and run CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM.

  2. Execute the CHANGE function and select menu item 7 to disable a quorum disk (see Section 8.4, “Changing Computer Characteristics”).

  3. Reboot the node (see Section 8.6.7, “Rebooting”).

All quorum disk watchers will be disabled.

Perform the following steps:
  1. Perform the preceding steps 1 and 2 for all computers with the quorum disk enabled.

  2. Reconfigure the cluster according to the instructions in Section 8.6, “Post-configuration Tasks”.

8.4. Changing Computer Characteristics

As your processing needs change, you may want to add satellites to an existing OpenVMS Cluster, or you may want to change an OpenVMS Cluster that is based on one interconnect (such as the CI or DSSI interconnect, or HSC subsystem) to include several interconnects.

Table 8.8, “CHANGE Options of the Cluster Configuration Procedure” describes the operations you can accomplish when you select the CHANGE option from the main menu of the cluster configuration command procedure.

Note: All operations except changing a satellite's LAN (Ethernet or FDDI) hardware address must be executed on the computer whose characteristics you want to change.
Table 8.8. CHANGE Options of the Cluster Configuration Procedure
OptionOperation Performed

Enable the local computer as a disk server

Loads the MSCP server by setting, in MODPARAMS.DAT, the value of the MSCP_LOAD parameter to 1 and the MSCP_SERVE_ALL parameter to 1 or 2.

Disable the local computer as a disk server

Sets MSCP_LOAD to 0.

Enable the local computer as a boot server

If you are setting up an OpenVMS Cluster that includes satellites, you must perform this operation once before you attempt to add satellites to the cluster. You thereby enable MOP service for the LAN adapter circuit that the computer uses to service operating system load requests from satellites. When you enable the computer as a boot server, it automatically becomes a disk server (if it is not one already) because it must serve its system disk to satellites.

Disable the local computer as a boot server

Disables DECnet MOP service for the computer's adapter circuit.

Enable IP for cluster communications on the local computer

Loads the port driver PEDRIVER by setting the value of theNISCS_LOAD_PEA0 parameter to 1 in MODPARAMS.DAT. Also, sets the value of NISCS_USE_UDP to 1 in MODPARAMS.DAT, which enables loading of the IP configuration files. Creates the cluster security database file, SYS$SYSTEM:[SYSEXE]CLUSTER_AUTHORIZE.DAT, on the local computer's system disk.

Disable IP for cluster communications on the local computer

Sets NISCS_USE_UDP to 0.

Enable the LAN for cluster communications on the local computer

Loads the port driver PEDRIVER by setting the value of the NISCS_LOAD_PEA0 parameter to 1 in MODPARAMS.DAT. Creates the cluster security database file, SYS$SYSTEM:[SYSEXE]CLUSTER_AUTHORIZE.DAT, on the local computer's system disk.

Caution: The VAXCLUSTER system parameter must be set to 2 if the NISCS_LOAD_PEA0 parameter is set to 1. This ensures coordinated access to shared resources in the cluster and prevents accidental data corruption.

Disable the LAN for cluster communications on the local computer

Sets NISCS_LOAD_PEA0 to 0.

Enable a quorum disk on the local computer

In MODPARAMS.DAT, sets the DISK_QUORUM system parameter to a device name; sets the value of QDSKVOTES to 1 (default value).

Disable a quorum disk on the local computer

In MODPARAMS.DAT, sets a blank value for the DISK_QUORUM system parameter; sets the value of QDSKVOTES to 1.

Change a satellite's LAN hardware address

Changes a satellite's hardware address if its LAN device needs replacement. Both the permanent and volatile network databases and NETNODE_UPDATE.COM are updated on the local computer.

Rule: You must perform this operation on each computer enabled as a boot server for the satellite.

Enable the local computer as a tape server

Loads the TMSCP server by setting, in MODPARAMS.DAT, the value of the TMSCP_LOAD parameter to 1 and the TMSCP_SERVE_ALL parameter to 1 or 2.

Disable the local computer as a tape server

Sets TMSCP_LOAD to zero.

Change the local computer's node allocation class value

Sets a value for the computer's ALLOCLASS parameter in MODPARAMS.DAT.

Change the local computer's tape allocation class value

Sets a value from 1 to 255 for the computer's TAPE_ALLOCLASS parameter in MODPARAMS.DAT. The default value is zero. You must specify a nonzero tape allocation class parameter if this node is locally connected to a dual-ported tape, or if it will be serving any multiple-host tapes (for example, TF nn or HSC connected tapes) to other cluster members. Satellites usually have TAPE_ALLOCLASS set to zero.

Change the local computer's port allocation class value

Sets a value for the computer's ALLOCLASS parameter in MODPARAMS.DAT for all devices attached to it.

Enable MEMORY CHANNEL

Sets MC_SERVICES_P2 to 1 to load the PMDRIVER (PMA0) cluster driver. This system parameter enables MEMORY CHANNEL on the local computer for node-to-node cluster communications.

Disable MEMORY CHANNEL

Sets MC_SERVICES_P2 to 0 so that the PMDRIVER (PMA0) cluster driver is not loaded. The setting of 0 disables MEMORYCHANNEL on the local computer as the node-to-node cluster communications interconnect.

8.4.1. Preparation

You usually need to perform a number of steps before using the cluster configuration command procedure to change the configuration of your existing cluster.

Table 8.9, “Tasks Involved in Changing OpenVMS Cluster Configurations” suggests several typical configuration changes and describes the procedures required to make them.
Table 8.9. Tasks Involved in Changing OpenVMS Cluster Configurations

Task

Procedure

Add satellite nodes

Perform these operations on the computer that will be enabled as a cluster boot server:
  1. Execute the CHANGE function to enable the first installed computer as a boot server (see Example 8.9, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Boot Server”).

  2. Execute the ADD function to add the satellite (as described in Section 8.2, “Adding Computers”).

  3. Reconfigure the cluster according to the post-configuration instructions in Section 8.6, “Post-configuration Tasks”.

Change an existing CI or DSSI cluster to include satellite nodes

To enable cluster communications over the LAN (Ethernet or FDDI) on all computers, and to enable one or more computers as boot servers, proceed as follows:
  1. Log in as system manager on each computer, invoke either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM, and execute the CHANGE function to enable LAN communications.

    Rule: You must perform this operation on all computers.

    Note: You must establish a cluster group number and password on all system disks in the OpenVMS Cluster before you can successfully add a satellite node using the CHANGE function of the cluster configuration procedure.

  2. Execute the CHANGE function to enable one or more computers as boot servers.

  3. Reconfigure the cluster according to the post-configuration instructions in Section 8.6, “Post-configuration Tasks”.

Change an existing LAN-based cluster to include CI and DSSI interconnects

Before performing the operations described here, be sure that the computers and HSC subsystems or RF disks you intend to include in your new configuration are correctly installed and checked for proper operation.

The method you use to include CI and DSSI interconnects with an existing LAN-based cluster configuration depends on whether your current boot server is capable of being configured as a CI or DSSI computer.

Note: The following procedures assume that the system disk containing satellite roots will reside on an HSC disk (for CI configurations) or an RF disk (for DSSI configurations).
  • If the boot server can be configured as a CI or DSSI computer, proceed as follows:
    1. Log in as system manager on the boot server and perform an image backup operation to back up the current system disk to a disk on an HSC subsystem or RF storage device. (For more information about backup operations, refer to the VSI OpenVMS System Management Utilities Reference Manual).

    2. Modify the computer's default bootstrap command procedure to boot the computer from the HSC or RF disk, according to the instructions in the appropriate system-specific installation and operations guide.

    3. Shut down the cluster. Shut down the satellites first, and then shut down the boot server.

    4. Boot the boot server from the newly created system disk on the HSC or RF storage subsystem.

    5. Reboot the satellites.

  • If your current boot server cannot be configured as a CI or a DSSI computer, proceed as follows:
    1. Shut down the old local area cluster. Shut down the satellites first, and then shut down the boot server.

    2. Install the OpenVMS operating system on the new CI computer's HSCsystem disk or on the new DSSI computer's RF disk, as appropriate. When the installation procedure asks whether you want to enable the LAN for cluster communications, answer YES.

    3. When the installation completes, log in as system manager, and configure and start the DECnet for OpenVMS network as described in Chapter 4, The OpenVMS Cluster Operating Environment.

    4. Execute the CHANGE function of either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to enable the computer as a boot server.

    5. Log in as system manager on the newly added computer and execute the ADD function of either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to add the former LAN cluster members (including the former boot server) as satellites.

  • Reconfigure the cluster according to the post-configuration instructions in Section 8.6, “Post-configuration Tasks”.

Convert a standalone computer to an OpenVMS Cluster computer

Execute either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM on a standalone computer to perform either of the following operations:
  • Add the standalone computer with its own system disk to an existing cluster.

  • Set up the standalone computer to form a new cluster if the computer was not set up as a cluster computer during installation of the operating system.

  • Reconfigure the cluster according to the post-configuration instructions in Section 8.6, “Post-configuration Tasks”.

See Example 8.13, “Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Convert a Standalone Computer to a Cluster Boot Server”, which illustrates the use of CLUSTER_CONFIG.COM on standalone computer PLUTO to convert PLUTO to a cluster boot server.

If your cluster uses DECdtm services, you must create a transaction log for the computer when you have configured it into your cluster. For step-by-step instructions on how to do this, see the chapter on DECdtm services in the VSI OpenVMS System Manager's Manual.

Enable or disable disk-serving or tape-serving functions

After invoking either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to enable or disable the disk or tape serving functions, run AUTOGEN with the REBOOT option to reboot the local computer (see Section 8.6.1, “Updating Parameter Files”).

Note: When the cluster configuration command procedure sets or changes values in MODPARAMS.DAT, the new values are always appended at the end of the file so that they override earlier values. You may want to edit the file occasionally and delete lines that specify earlier values.

8.4.2. Examples

Examples 8.7 through 8.13 illustrate the use of CLUSTER_CONFIG_LAN.COM to perform the following operations:

Example 8.7. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Disk Server
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 5

Enter a value for BHAGAT's ALLOCLASS parameter [1]:
    The configuration procedure has completed successfully.

    BHAGAT has been enabled as a disk server. In MODPARAMS.DAT:

           MSCP_LOAD has been set to 1
           MSCP_SERVE_ALL has been set to 2

    Please run AUTOGEN to reboot BHAGAT:

           $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

    If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the
    cluster, using the procedure described in the OpenVMS Cluster Systems manual.

Example 8.8. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change the Local Computer's ALLOCLASS Value
$ @CLUSTER_CONFIG_LAN.COM
               Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 7

Enter a value for BHAGAT's ALLOCLASS parameter [1]: 2
    The configuration procedure has completed successfully.

    Since you have changed BHAGAT's ALLOCLASS value, you must reconfigure
    the cluster, using the procedure described in the "OpenVMS Cluster
    Systems" manual. This includes running AUTOGEN for BHAGAT as
    shown below, before rebooting the cluster:

           $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

    If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the
    cluster, using the procedure described in the OpenVMS Cluster Systems manual.

Example 8.9. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Boot Server
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 1

Enter a value for BHAGAT's ALLOCLASS parameter [1]: [Return]
    The configuration procedure has completed successfully.

    BHAGAT has been enabled as a boot server. Disk serving and
    LAN capabilities are enabled automatically. If BHAGAT was
    not previously set up as a disk server, please run AUTOGEN
    to reboot BHAGAT:

           $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

    If you have changed BHAGAT's ALLOCLASS value, you must reconfigure the
    cluster, using the procedure described in the OpenVMS Cluster Systems manual.

Example 8.10. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change a Satellite's Hardware Address
$ @CLUSTER_CONFIG_LAN.COM
             Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 11

What is the node's SCS node name? gomthi
    Note: The current hardware address entry for GOMTHI is 00-30-6E-4C-BB-1A.
What is GOMTHI's new LAN adapter hardware address? 00-30-6E-4C-BA-2A
    The configuration procedure has completed successfully.

Example 8.11. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Enable the Local Computer as a Tape Server
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 8

Enter a value for BHAGAT's TAPE_ALLOCLASS parameter [0]: [Return]
Should BHAGAT serve any tapes it sees, local and remote [Y]? [Return]

    BHAGAT has been enabled as a tape server. In MODPARAMS.DAT,
        TMSCP_LOAD has been set to 1
        TMSCP_SERVE_ALL has been set to 1

    Please run AUTOGEN to reboot BHAGAT:

           $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

    If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure
    the cluster, using the procedure described in the "OpenVMS Cluster
    Systems" manual. 

Example 8.12. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Change the Local Computer's TAPE_ALLOCLASS Value
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 3

CHANGE Menu

   1. Enable BHAGAT as a boot server.
   2. Disable BHAGAT as a boot server.
   3. Enable a quorum disk for BHAGAT.
   4. Disable a quorum disk for BHAGAT.
   5. Enable BHAGAT as a disk server.
   6. Disable BHAGAT as a disk server.
   7. Change BHAGAT's ALLOCLASS value.
   8. Enable BHAGAT as a tape server.
   9. Disable BHAGAT as a tape server.
  10. Change BHAGAT's TAPE_ALLOCLASS value.
  11. Change an IA64 satellite node's LAN adapter hardware address.
  12. Enable Cluster Communication using IP on BHAGAT.
  13. Disable Cluster Communication using IP on BHAGAT.
  14. Change BHAGAT's shared SCSI port allocation class value.
  15. Reset an IA64 satellite node's boot environment file protections.
  16. Return to MAIN menu.

Enter choice [16]: 10

Enter a value for BHAGAT's TAPE_ALLOCLASS parameter [0]: 1

    If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure
    the cluster, using the procedure described in the "OpenVMS Cluster
    Systems" Manual. This includes running AUTOGEN for BHAGAT as
    shown below, before rebooting the cluster:

           $ @SYS$UPDATE:AUTOGEN GETDATA REBOOT

    If you have changed BHAGAT's TAPE_ALLOCLASS value, you must reconfigure
    the cluster, using the procedure described in the OpenVMS Cluster Systems
    manual.

Example 8.13. Sample Interactive CLUSTER_CONFIG_LAN.COM Session to Convert a Standalone Computer to a Cluster Boot Server
$ @CLUSTER_CONFIG_LAN.COM
IA64 platform support is in procedure CLUSTER_CONFIG_LAN.COM.
    The currently running procedure, CLUSTER_CONFIG.COM, will call
    it for you.
                   Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    This IA64 node is not currently a cluster member.

MAIN Menu

   1. ADD MOON to existing cluster, or form a new cluster.
   2. MAKE a directory structure for a new root on a system disk.
   3. DELETE a root from a system disk.
   4. EXIT from this procedure.

Enter choice [4]: 1
Is the node to be a clustered node with a shared SCSI/FIBRE-CHANNEL bus (Y/N)? N

What is the node's SCS node name? moon

    DECnet is running on this node. Even though you are configuring a LAN-
    based cluster, the DECnet database will provide some information and
    may be updated.

Do you want to define a DECnet synonym [Y]? N
    IA64 node, using LAN for cluster communications.  PEDRIVER will be loaded.
    No other cluster interconnects are supported for IA64 nodes.
Enter this cluster's group number: 123
Enter this cluster's password:
Re-enter this cluster's password for verification:

Will MOON be a boot server [Y]? [Return]

        TCP/IP BOOTP and TFTP services must be enabled on IA64 boot nodes.

        Use SYS$MANAGER:TCPIP$CONFIG.COM on MOON to enable BOOTP and TFTP services
        after MOON has booted into the cluster.

Enter a value for MOON's ALLOCLASS parameter [0]:[Return]
Does this cluster contain a quorum disk [N]? [Return]

    The EXPECTED_VOTES system parameter of members of a cluster indicates the
    total number of votes present when all cluster members are booted, and is
    used to determine the minimum number of votes (QUORUM) needed for cluster
    operation.

EXPECTED_VOTES value for this cluster: 1

Warning:  Setting EXPECTED_VOTES to 1 allows this node to boot without
          being able to see any other nodes in the cluster.  If there is
          another instance of the cluster in existence that is unreachable
          via SCS but shares common drives (such as a Fibrechannel fabric)
          this may result in severe disk corruption.

Do you wish to re-enter the value of EXPECTED_VOTES [Y]? N

    The use of a quorum disk is recommended for small clusters to maintain
    cluster quorum if cluster availability with only a single cluster node is
    a requirement.

    For complete instructions, check the section on configuring a cluster
    in the "OpenVMS Cluster Systems" manual.


  WARNING: MOON will be a voting cluster member. EXPECTED_VOTES for
           this and every other cluster member should be adjusted at
           a convenient time before a reboot. For complete instructions,
           check the section on configuring a cluster in the "OpenVMS
           Cluster Systems" manual.

    Execute AUTOGEN to compute the SYSGEN parameters for your configuration
    and reboot MOON with the new parameters. This is necessary before
    MOON can become a cluster member.

Do you want to run AUTOGEN now [Y]? [Return]
    Running AUTOGEN -- Please wait.

%AUTOGEN-I-BEGIN, GETDATA phase is beginning.
.
.
. 

8.5. Creating a Duplicate System Disk

As you continue to add Integrity servers running on a common Integrity common system disk, or Alpha computers running on an Alpha common system disk, you eventually reach the disk's storage or I/O capacity. In that case, you want to add one or more common system disks to handle the increased load.

Reminder: Remember that a system disk cannot be shared between two architectures. Furthermore, you cannot create a system disk for one architecture from a system disk of a different architecture.

8.5.1. Preparation

You can use either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to set up additional system disks. After you have coordinated cluster common files as described in Chapter 5, Preparing a Shared Environment, proceed as follows:
  1. Locate an appropriate scratch disk for use as an additional system disk.

  2. Log in as system manager.

  3. Invoke either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM and select the CREATE option.

8.5.2. Example

As shown in Example 8.14, “Sample Interactive CLUSTER_CONFIG_LAN.COM CREATE Session”, the cluster configuration command procedure:
  1. Prompts for the device names of the current and new system disks.

  2. Backs up the current system disk to the new one.

  3. Deletes all directory roots (except SYS0) from the new disk.

  4. Mounts the new disk clusterwide.

Note: OpenVMS RMS error messages are displayed while the procedure deletes directory files. You can ignore these messages.
Example 8.14. Sample Interactive CLUSTER_CONFIG_LAN.COM CREATE Session
$ @CLUSTER_CONFIG_LAN.COM
              Cluster/IPCI Configuration Procedure
                   CLUSTER_CONFIG_LAN Version V2.84
                     Executing on an IA64 System

    DECnet-Plus is installed on this node.
    IA64 satellites will use TCP/IP BOOTP and TFTP services for downline loading.
    TCP/IP is installed and running on this node.

        Enter a "?" for help at any prompt.  If you are familiar with
        the execution of this procedure, you may want to mute extra notes
        and explanations by invoking it with "@CLUSTER_CONFIG_LAN BRIEF".

    BHAGAT is an IA64 system and currently a member of a cluster
    so the following functions can be performed:

MAIN Menu

   1. ADD an IA64 node to the cluster.
   2. REMOVE a node from the cluster.
   3. CHANGE a cluster member's characteristics.
   4. CREATE a duplicate system disk for BHAGAT.
   5. MAKE a directory structure for a new root on a system disk.
   6. DELETE a root from a system disk.
   7. EXIT from this procedure.

Enter choice [7]: 4

    The CREATE function generates a duplicate system disk.

            o It backs up the current system disk to the new system disk.

            o It then removes from the new system disk all system roots.

  WARNING: Do not proceed unless you have defined appropriate logical names
           for cluster common files in SYLOGICALS.COM.  For instructions,
           refer to the "OpenVMS Cluster Systems" manual.

Do you want to continue [N]? Y

    This procedure will now ask you for the device name of the current
    system disk. The default device name (DISK$BHAGAT_SYS:) is the logical
    volume name of SYS$SYSDEVICE:.

What is the device name of the current system disk [DISK$BHAGAT_SYS:]?
What is the device name of the new system disk?
.
.
. 

8.6. Post-configuration Tasks

Some configuration functions, such as adding or removing a voting member or enabling or disabling a quorum disk, require one or more additional operations.

These operations are listed in Table 8.10, “Actions Required to Reconfigure a Cluster” and affect the integrity of the entire cluster. Follow the instructions in the table for the action you should take after executing either CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM to make major configuration changes.
Table 8.10. Actions Required to Reconfigure a Cluster
After running the cluster configuration procedure to...You should...

Add or remove a voting member

Update the AUTOGEN parameter files and the current system parameter files for all nodes in the cluster, as described in Section 8.6.1, “Updating Parameter Files”.

Enable a quorum disk

Perform the following steps:
  1. Update the AUTOGEN parameter files and the current system parameter files for all quorum watchers in the cluster, as described in Section 8.6.1, “Updating Parameter Files”.

  2. Reboot the nodes that have been enabled as quorum disk watchers (Section 2.3.9, “Quorum Disk Watcher”).

Reference: See also Section 8.2.4, “Adding a Quorum Disk” for more information about adding a quorum disk.

Disable a quorum disk

Perform the following steps:

Caution: Do not perform these steps until you are ready to reboot the entire OpenVMS Cluster system. Because you are reducing quorum for the cluster, the votes cast by the quorum disk being removed could cause cluster partitioning.
  1. Update the AUTOGEN parameter files and the current system parameter files for all quorum watchers in the cluster, as described in Section 8.6.1, “Updating Parameter Files”.

  2. Evaluate whether or not quorum will be lost without the quorum disk:

Quorum will not be lost

THEN...

Quorum will not be lost

Perform these steps:
  1. Use the DCL command SET CLUSTER/EXPECTED_VOTES to reduce the value of quorum.

  2. Reboot the nodes that have been disabled as quorum disk watchers. (Quorum disk watchers are described in Section 2.3.9, “Quorum Disk Watcher”.)

Quorum will be lost

Shut down and reboot the entire cluster.

Reference: Cluster shutdown is described in Section 8.6.2, “Shutting Down the Cluster”.

Reference: See also Section 8.3.2, “Removing a Quorum Disk” for more information about removing a quorum disk.

Add a satellite node

Perform these steps:

Enable or disable the LAN or IP for cluster communications

Update the current system parameter files and reboot the node on which you have enabled or disabled the LAN or IP (Section 8.6.1, “Updating Parameter Files”).

Change allocation class values

Update the current system parameter files and shut down and reboot the entire cluster (Sections 8.6.1 and 8.6.2).

Change the cluster group number or password

Shut down and reboot the entire cluster (Sections 8.6.2 and 8.6.7).

8.6.1. Updating Parameter Files

The cluster configuration command procedures (CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM) can be used to modify parameters in the AUTOGEN parameter file for the node on which it is run.

In some cases, such as when you add or remove a voting cluster member, or when you enable or disable a quorum disk, you must update the AUTOGEN files for all the other cluster members.

Use either of the methods described in the following table.

Method

Description

Update MODPARAMS.DAT files

Edit MODPARAMS.DAT in all cluster members' [SYS x.SYSEXE] directories and adjust the value for the EXPECTED_VOTES system parameter appropriately.

For example, if you add a voting member or if you enable a quorum disk, you must increment the value by the number of votes assigned to the new member (usually 1). If you add a voting member with one vote and enable a quorum disk with one vote on that computer, you must increment the value by 2.

Update AGEN$ files

Update the parameter settings in the appropriate AGEN$ include files:
  • For satellites, edit SYS$MANAGER:AGEN$NEW_SATELLITE_DEFAULTS.DAT.

  • For nonsatellites, edit

    SYS$MANAGER:AGEN$NEW_NODE_DEFAULTS.DAT.

Reference: These files are described in Section 8.2.2, “Common AUTOGEN Parameter Files”.

You must also update the current system parameter files (IA64VMSSYS.PAR or ALPHAVMSSYS.PAR, as appropriate) so that the changes take effect on the next reboot.

Use either of the methods described in the following table.

Method

Description

SYSMAN utility

Perform the following steps:
  1. Log in as system manager.

  2. Run the SYSMAN utility to update the EXPECTED_VOTES system parameter on all nodes in the cluster. For example:
    $ RUN SYS$SYSTEM:SYSMAN
    %SYSMAN-I-ENV, current command environment:
      Clusterwide on local cluster
      Username SYSTEM  will be used on nonlocal nodes
    
    SYSMAN> SET ENVIRONMENT/CLUSTER
    SYSMAN> PARAM USE CURRENT
    SYSMAN> PARAM SET EXPECTED_VOTES 2
    SYSMAN> PARAM WRITE CURRENT
    SYSMAN> EXIT

AUTOGEN utility

Perform the following steps:
  1. Log in as system manager.

  2. Run the AUTOGEN utility to update the EXPECTED_VOTES system parameter on all nodes in the cluster. For example:
    $ RUN SYS$SYSTEM:SYSMAN
    %SYSMAN-I-ENV, current command environment:
      Clusterwide on local cluster
      Username SYSTEM will be used on nonlocal nodes
    
    SYSMAN> SET ENVIRONMENT/CLUSTER
    SYSMAN> DO @SYS$UPDATE:AUTOGEN GETDATA SETPARAMS
    SYSMAN> EXIT

Do not specify the SHUTDOWN or REBOOT option.

Hints: If your next action is to shut down the node, you can specify SHUTDOWN or REBOOT (in place of SETPARAMS) in the DO @SYS$UPDATE:AUTOGEN GETDATA command.

Both of these methods propagate the values to the computer's ALPHAVMSSYS.PAR file on Alpha computers or to the IA64VMSSYS.PAR file on Integrity server systems. In order for these changes to take effect, continue with the instructions in either Section 8.6.2, “Shutting Down the Cluster” to shut down the cluster or in Section 8.6.3, “Shutting Down a Single Node” to shut down the node.

8.6.2. Shutting Down the Cluster

Using the SYSMAN utility, you can shut down the entire cluster from a single node in the cluster. Follow these steps to perform an orderly shutdown:
  1. Log in to the system manager's account on any node in the cluster.

  2. Run the SYSMAN utility and specify the SET ENVIRONMENT/CLUSTER command. Be sure to specify the /CLUSTER_SHUTDOWN qualifier to the SHUTDOWN NODE command. For example:


$ RUN SYS$SYSTEM:SYSMAN
SYSMAN> SET ENVIRONMENT/CLUSTER
%SYSMAN-I-ENV, current command environment:
  Clusterwide on local cluster
  Username SYSTEM will be used on nonlocal nodes

SYSMAN> SHUTDOWN NODE/CLUSTER_SHUTDOWN/MINUTES_TO_SHUTDOWN=5 -
_SYSMAN> /AUTOMATIC_REBOOT/REASON="Cluster Reconfiguration"
%SYSMAN-I-SHUTDOWN, SHUTDOWN request sent to node
%SYSMAN-I-SHUTDOWN, SHUTDOWN request sent to node
SYSMAN>

SHUTDOWN message on BHAGAT from user SYSTEM at BHAGAT Batch 11:02:10
BHAGAT will shut down in 5 minutes; back up shortly via automatic reboot.
Please log off node BHAGAT.
Cluster Reconfiguration
SHUTDOWN message on BHAGAT from user SYSTEM at BHAGAT Batch 11:02:10
PLUTO will shut down in 5 minutes; back up shortly via automatic reboot.
Please log off node PLUTO.
Cluster Reconfiguration

For more information, see Section 10.6, “Shutting Down a Cluster”.

8.6.3. Shutting Down a Single Node

To stop a single node in an OpenVMS Cluster, you can use either the SYSMAN SHUTDOWN NODE command with the appropriate SETENVIRONMENT command or the SHUTDOWN command procedure. These methods are described in the following table.

Method

Description

SYSMAN utility

Follow these steps:
  1. Log in to the system manager's account on any node in the OpenVMS Cluster.

  2. Run the SYSMAN utility to shut down the node, as follows:
    $ RUN SYS$SYSTEM:SYSMAN
    SYSMAN> SET ENVIRONMENT/NODE=JUPITR
    Individual nodes: JUPITR
    Username SYSTEM will be used on nonlocal nodes
    
    SYSMAN> SHUTDOWN NODE/REASON="Maintenance" -
    _SYSMAN> /MINUTES_TO_SHUTDOWN=5
    Hint: To shut down a subset of nodes in the cluster, you can enter several node names separated by commas on the SET ENVIRONMENT/NODE command. The following command shuts down nodes JUPITR and SATURN:
    SYSMAN> SET ENVIRONMENT/NODE=(JUPITR,SATURN)

SHUTDOWN command procedure

Follow these steps:
  1. Log in to the system manager's account on the node to be shut down.

  2. Invoke the SHUTDOWN command procedure as follows:
    $ @SYS$SYSTEM:SHUTDOWN

For more information, see Section 10.6, “Shutting Down a Cluster”.

8.6.4. Updating Network Data

Whenever you add a satellite, the cluster configuration command procedure you use (CLUSTER_CONFIG_LAN.COM or CLUSTER_CONFIG.COM) updates both the permanent and volatile remote node network databases (NETNODE_REMOTE.DAT) on the boot server. However, the volatile databases on other cluster members are not automatically updated.

To share the new data throughout the cluster, you must update the volatile databases on all other cluster members. Log in as system manager, invoke the SYSMAN utility, and enter the following commands at the SYSMAN> prompt:
$ RUN SYS$SYSTEM:SYSMAN
SYSMAN> SET ENVIRONMENT/CLUSTER
%SYSMAN-I-ENV, current command environment:
        Clusterwide on lo