Print | Rate this content

HP-UX Serviceguard - Method to change the cluster lock VG or disk

Information

What is the procedure to change the cluster lock disk, or cluster lock VG and disk?

Details

Split brain Arbitration concept

When exactly half of the nodes of an active cluster cannot interchange heartbeat messages with the other half of the nodes, a split-brain situation is eminent. To avoid the creation of two separate half clusters that may operate the same packages, Serviceguard causes all nodes to arbitrate with the cluster lock disk (or quorum server). The half-set of nodes that contact the lock disk first are allowed to continue to operate the cluster. The half-set that contact the lock disk later determine these are late, and are required to TOC (perform a memory dump and reboot to preserve data integrity. Then the first half-set sees the late half-set leave the cluster, and take over any packages that were running on them (if configured do so by the administrator).

The cluster binary file /etc/cmcluster/cmclconfig contain the cluster configuration that is loaded into memory when a node starts the main Serviceguard daemon cmcld. The binary file content is created or updated by a successful cmapplyconf operation on a cluster configuration file.

To use cluster lock disk arbitration, the cluster lock information is configured by the FIRST_CLUSTER_LOCK_VG parameter and a FIRST_CLUSTER_LOCK_PV parameters in the cluster configuration file.

This file is used in the creation of the cluster binary file. The example commands below show the cluster lock features recorded in the cluster binary file.

For Serviceguard versions prior to A.11.18:


# cmviewconf | grep -i -e "node name" -e lock
flags: 12 (single cluster lock)
first lock vg name: /dev/vg01
second lock vg name: (not configured)
Node name: eon
first lock pv name: /dev/dsk/c0t4d0
first lock disk interface type: scsi3
Node name: ion
first lock pv name: /dev/dsk/c0t4d0
first lock disk interface type: scsi3

For A.11.18 and later:


# cmviewcl -v -f line | grep lock
package:dump|node:rxg16u09|state=blocked
first_cluster_lock|volume_group=/dev/vglock
first_cluster_lock|node:rxg16u07|physical_volume=/dev/dsk/c22t0d1
first_cluster_lock|node:rxg16u07|status=up
first_cluster_lock|node:rxg16u07|interface_type=fcd_vbus
first_cluster_lock|node:rxg16u09|physical_volume=/dev/dsk/c22t0d1
first_cluster_lock|node:rxg16u09|status=up
first_cluster_lock|node:rxg16u09|interface_type=fcd_vbus

The FIRST_CLUSTER_LOCK_PV reference for each node must use the device file each server uses to communicate with that lock disk regardless of whether the device file name is the same or not.

Online change:

Serviceguard A.11.19 and later versions allow the root user to change the cluster lock VG/PV parameters while the cluster is running. Earlier versions of Serviceguard will not allow cmapplyconf to change cluster lock values in the cluster binary file unless the cluster is halted.

If any of the FIRST_CLUSTER_LOCK_* values must be changed, then follow the procedure listed below:

  1. Locate the cluster configuration ASCII file. If it is whereabouts is unknown, recreate it on one of the nodes in the cluster:

    $ cd /etc/cmcluster

    $ cmgetconf cluster.config

  2. Insure the cluster.config reflects the current hardware and network configuration:

    $ cmcheckconf -C cluster.config

    If this command fails for other reasons other than a missing cluster lock disk, the cluster.config may be out-dated. Either correct the problem, or dissolve (cmdeleteconf -f) the current cluster (when the cluster is halted) and create a fresh cluster.config using cmquerycl.

  3. Locate the FIRST_CLUSTER_LOCK_VG and FIRST_CLUSTER_LOCK_PV references in the cluster.config file. There is one FIRST_CLUSTER_LOCK_PV parameter per node (section).

  4. Since the cluster lock disk structures are not in the user data space, the admin can designate any LVM volume group that is shared by each node for the cluster lock function, including a VG that is used by a Serviceguard package. It is not necessary to dedicate a VG for cluster lock unless it is the only LVM VG shared by the nodes in the cluster.

    The FIRST_CLUSTER_LOCK_VG volume group must be listed in /etc/lvmtab or /etc/lvmtab_p on all nodes. If it is not listed on one more nodes, vgimport it using this method:

    On the node where the VG already exists:

    Identify whether the VG is an LVM1 or LVM2 volume group.

    Example:


    $ vgchange -a e <vg_name>
    $ vgdisplay <vg_name> | grep Version
    VG Version 1.0 # uses group major number 64
    VG Version 2.0 # uses group major number 128

    Create a map file of the volume group. The map file contains the VGID, and the unique logical volume names.


    $ vgexport -pvs -m /etc/lvmconf/<vg_name>.map /dev/<vg_name>
    $ scp /etc/lvmconf/<vg_name>.map {OTHERNODE}:/etc/lvmconf/

    On the node where the VG will be imported:

    $ ll /dev/*/group

    Identify the next unused minor number:

    $ mkdir /dev/<vg_name>

    Using the result of the vgdisplay above, load the correct major number in the following command:

    $ mknod /dev/<vg_name>/group c MM 0xNN0000

    Where MM= major number, NN=next unused minor number

  5. Update the FIRST_CLUSTER_LOCK_VG reference and/or FIRST_CLUSTER_LOCK_PV references in the file.

    Example:

    FIRST_CLUSTER_LOCK_VG /dev/vg03

  6. If needed, halt the cluster.

    $ cmhaltcl -f

  7. Activate the cluster lock VG.

    NOTE: If the lock VG is already owned by the cluster, it must first be declustered before it can be activated.

    Example:

    $ vgchange -c n vg03

    $ vgchange -a y vg03

  8. Use cmapplyconf (which includes cmcheckconf) to update the cluster binary file.


    $ cmapplyconf -f -C cluster.config
    Begin cluster verification...
    Modifying node <nodeA> in cluster <ClusterName>
    Modifying node <nodeB> in cluster <ClusterName>

    It should not be necessary to delete the cluster using cmdeleteconf prior to changing the cluster lock volume group or disk.

  9. Deactivate the cluster lock VG:

    $ vgchange -a n vg03

  10. Start the cluster if halted:

    $ cmruncl

Legal Disclaimer: Products sold prior to the November 1, 2015 separation of Hewlett-Packard Company into Hewlett Packard Enterprise Company and HP Inc. may have older product names and model numbers that differ from current models.

Provide feedback

Please rate the information on this page to help us improve our content. Thank you!
Document title: HP-UX Serviceguard - Method to change the cluster lock VG or disk
Document ID: emr_na-c03383413-1
How helpful was this document?
How can we improve this document?
Note: Only English language comments can be accepted at this time.
Please wait while we process your request.