How to Configure HVR for High Availability in Oracle Cluster
Question
How can I configure the HVR services for high availability (HA) in a Cluster?
Environment
HVR 5
Answer
One of the advantages of a clustered environment is the ability to survive a node failure. Individual HVR processes always run on only one of the nodes at the time.
This article applies to Unix and Linux environments only. On Windows, use Windows Services in cluster mode to enable high availability for HVR services in the cluster.
HVR runs two daemons that start other processes:
An agent installation of HVR will only have the hvrremotelistener. A hub installation of HVR will have a hvrscheduler and (especially on Linux and Unix) in many cases also a hvrremotelistener defined.
In order for the hvrscheduler to be highly available in a cluster, the $HVR_CONFIG directory must be shared between the nodes, and the UID for the operating system user that runs HVR must be identical on every node. Point $HVR_CONFIG directory to a shared file system, which can be NAS or clustered. Note that the availability of the shared file system is essential for HVR's availability in the cluster. If an NFS share hosted by one of the nodes is used as the shared file system, its availability becomes dependent on that specific node.
The HVR binaries can be installed on a shared file system or on individual nodes. A single shared installation is easier to manage and takes up less storage space. However, a shared storage system may perform slower than a storage system that is not shared, and this may or may not impact HVR's processing depending on the transaction mix that must be captured. Currently, rolling upgrade capability is not available for HVR, which entirely eliminates downtime.
Use file $HVR_HOME/lib/hvr_boot to make HVR highly available in a cluster environment. Copy the file $HVR_HOME/lib/hvr_boot and create a hvrtab file for every daemon that must be made highly available in the cluster. The copies of the files hvr_boot and hvrtab must be in the same location on every node in the cluster and may be stored on a shared file system for easier setup and maintenance. For example, if a cluster acts as a hub and runs the hvrremotelistener, then there must be two copies of the file hvr_boot (e.g. file hvr_boot_scheduler and file hvr_boot_listener4343) and two copies of the file hvrtab (e.g. file hvrtab_scheduler and file hvrtab_listener4343). The respective hvrtab files must contain the correct line to invoke the service.
For example, the contents of the file hvrtab_scheduler:
#hvrtab_scheduler to start the HVR scheduler in an Oracle RAC environment as OS user hvr hvr hvrhub/!{Qb/Q.KqR}!@rac /mnt/shared/hvr/hvr_home /mnt/shared/hvr/hvr_config -EORACLE_HOME=<path to oracle_home>
For example, the contents of the file hvrtab_listener4343:
#hvrtab_listener4343 to start the HVR listener on port 4343 in a clustered environment root 4343 /mnt/shared/hvr/hvr_home /mnt/shared/hvr/hvr_config -EORACLE_HOME=<path to oracle_home>
Change the copies of the hvr_boot file to refer to the correct hvrtab files (at the beginning of the file).
For example in the file hvr_boot_scheduler:
HVRTAB=/mnt/shared/hvr/scripts/hvrtab_scheduler
For example in the file hvr_boot_listener4343:
HVRTAB=/mnt/shared/hvr/scripts/hvrtab_listener4343
The HVR scheduler must be registered as a cluster resource in the Clusterware software because it should only be run on one of the nodes at the time. Use Clusterware commands to do this logged in as the root user (on any one of the nodes).
Example commands for HVR Scheduler Service
To create the resource in Oracle Clusterware, log in as root with the environment set to use the Oracle Grid binaries and execute the following command:
[root@ol5-112-rac1 ~] crsctl add resource hvrschdlr
Sample output:
-type cluster_resource -attr "ACTION_SCRIPT=/mnt/shared/hvr/scripts/hvr_boot_scheduler, CHECK_INTERVAL=30"
To start the resource, use:
[root@ol5-112-rac1 ~]# crsctl start resource hvrschdlr
Sample output:
CRS-2672: Attempting to start 'hvrschdlr' on 'ol5-112-rac2' CRS-2676: Start of 'hvrschdlr' on 'ol5-112-rac2' succeeded
To relocate the resource to another node, use:
[root@ol5-112-rac1 ~]# crsctl relocate resource hvrschdlr
Sample output:
CRS-2673: Attempting to stop 'hvrschdlr' on 'ol5-112-rac2' CRS-2677: Stop of 'hvrschdlr' on 'ol5-112-rac2' succeeded CRS-2672: Attempting to start 'hvrschdlr' on 'ol5-112-rac1' CRS-2676: Start of 'hvrschdlr' on 'ol5-112-rac1' succeeded
To check the status of the resource, use:
[root@ol5-112-rac1 ~]# crsctl status resource hvrschdlr
Sample output:
NAME=hvrschdlr TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on ol5-112-rac1
To stop the resource, use:
[root@ol5-112-rac1 ~]# crsctl stop resource hvrschdlr
Sample output:
CRS-2673: Attempting to stop 'hvrschdlr' on 'ol5-112-rac1' CRS-2677: Stop of 'hvrschdlr' on 'ol5-112-rac1' succeeded
To clean up the resource, use:
[root@ol5-112-rac1 ~]# crsctl delete resource hvrschdlr
The HVR listener must be registered as a local resource to run on every node in the cluster. Use Clusterware commands to do this logged in as the root user (on any one of the nodes).
Example commands for HVR Listener Service
To create the resource in Oracle Clusterware, log in as root with the environment set to use the Oracle Grid binaries and execute the following command:
[root@ol5-112-rac1 ~]# crsctl add resource hvrlsnr
Sample output:
-type local_resource -attr "ACTION_SCRIPT=/mnt/shared/hvr/scripts/hvr_boot_listener4343, CHECK_INTERVAL=30"
To start the resource, use:
[root@ol5-112-rac1 lib]# crsctl start resource hvrlsnr
Sample output:
CRS-2672: Attempting to start 'hvrlsnr' on 'ol5-112-rac1' CRS-2672: Attempting to start 'hvrlsnr' on 'ol5-112-rac2' CRS-2676: Start of 'hvrlsnr' on 'ol5-112-rac1' succeeded CRS-2676: Start of 'hvrlsnr' on 'ol5-112-rac2' succeeded
To check the status of the resource, use:
[root@ol5-112-rac1 ~]# crsctl status resource hvrlsnr
Sample output:
NAME=hvrlsnr TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on ol5-112-rac1
To stop the resource, use:
[root@ol5-112-rac1 ~]# crsctl stop resource hvrlsnr
Sample output:
CRS-2673: Attempting to stop 'hvrlsnr' on 'ol5-112-rac1' CRS-2677: Stop of 'hvrlsnr' on 'ol5-112-rac1' succeeded
To clean up the resource, use:
[root@ol5-112-rac1 ~]# crsctl delete resource hvrlsnr