How To Configure Online Failover/Failback on CentOS 6 Using Heartbeat

This article will explain how to configuration failover/failback using Heartbeat application. According this article : http://linux-ha.org/wiki/Heartbeat

Heartbeat is a daemon that provides cluster infrastructure (communication and membership) services to its clients. This allows clients to know about the presence (or disappearance!) of peer processes on other machines and to easily exchange messages with them.

In this guidance, i am build 2 system for online failover. The systems using CentOS 6 64 Bit. For easy understanding, this is my information system

# Server 1
Hostname   : node1
Domain     : imanudin.net
IP Address : 192.168.80.91

# Server 2
Hostname   : node2
Domain     : imanudin.net
IP Address : 192.168.80.92

# Alias IP for online failover testing
IP Address : 192.168.80.93

# Configure Network
First, we must configure network on CentOS. Assuming name of your network interface is eth0. Do the following configuration on all nodes (node1 and node2) and adjust on node2

1.vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
IPADDR=192.168.80.91
NETMASK=255.255.255.0
DNS1=192.168.80.91
GATEWAY=192.168.80.11
DNS2=192.168.80.11
DNS3=8.8.8.8
USERCTL=no

Restart network service and setup for automatic boot on all nodes (node1 and node2)

1.service network restart

2.chkconfig network on

# Configure Disable Selinux & Firewall on all nodes (node1 and node2)
Open file /etc/sysconfig/selinux and change SELINUX=enforcing become SELINUX=disabled. Also disable some service such as iptables and ip6tables.

1.setenforce 0

2.service iptables stop

3.service ip6tables stop

4.chkconfig iptables off

5.chkconfig ip6tables off

# Configure /etc/hosts and hostname on all nodes (node1 and node2)
Open file /etc/hosts and configure as follows

# node1
127.0.0.1     localhost
192.168.80.91 node1.imanudin.net node1
192.168.80.92 node2.imanudin.net node2

# node2
127.0.0.1     localhost
192.168.80.91 node1.imanudin.net node1
192.168.80.92 node2.imanudin.net node2

Do the following command as root and open file /etc/sysconfig/network to change hostname
– On node1

1.hostname node1.imanudin.net

2.vi /etc/sysconfig/network

Change HOSTNAME so that like below :

NETWORKING=yes
HOSTNAME=node1.imanudin.net

– On node2

1.hostname node2.imanudin.net

2.vi /etc/sysconfig/network

Change HOSTNAME so that like below :

NETWORKING=yes
HOSTNAME=node2.imanudin.net

# Update repos and install packages Heartbeat on all nodes (node1 and node2)

1.yum update

2.yum install epel-release

3.yum -y install heartbeat

If you cannot get epel repo, please use this repo and install : http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
# Configure Heartbeat
– Create a file /etc/ha.d/ha.cf (enough on node1 only)

1.vi /etc/ha.d/ha.cf

fill with the following line

keepalive 2
warntime 5
deadtime 15
initdead 90
udpport 694
auto_failback on
ucast eth0 192.168.80.92
logfile /var/log/ha-log
node node1.imanudin.net node2.imanudin.net

Note :
eth0 is interface on your system. If your system using eth1 for interface name, please change eth0 to the eth1. 192.168.80.92 is IP Address of node2
– Create a file /etc/ha.d/authkeys (enough on node1 only)

1.vi /etc/ha.d/authkeys

fill with the following line

auth 2
2 crc

change permission authkeys

1.chmod 0600 /etc/ha.d/authkeys

– Create a file /etc/ha.d/haresources (enough on node1 only)

1.vi /etc/ha.d/haresources

fill with the following line

node1.imanudin.net IPaddr::192.168.80.93/24/eth0:0

Note :
node1.imanudin.net will become as a master server. 192.168.80.93 is an alias IP for testing online/failover
# Copy those files from node1 to node2 (run the following command on node1)

1.cd /etc/ha.d/

2.scp authkeys ha.cf haresources root@192.168.80.92:/etc/ha.d/

# Change ha.cf file on node2 (run the following command on node2)

1.vi /etc/ha.d/ha.cf

change line ucast eth0 192.168.80.92 so that become

ucast eth0 192.168.80.91

192.168.80.91 is IP Address of node1
# Start service Heartbeat and configure for automatic startup at boot on all nodes (node1 and node2)

1.service heartbeat start

2.chkconfig heartbeat on

TESTING ONLINE FAILOVER/FAILBACK
After your start service heartbeat on all nodes, you will see an alias IP on node1. Please check with command ifconfig.For testing failover, please stop service heartbeat on node1 (service heartbeat stop). Please check your IP on node2 with command ifconfig. You will see an alias IP on node2 (an alias IP that has been taken by node2). For testing failback, please start again service heartbeat on node1 (service heartbeat start). An alias IP will automatically taken by node1.
TESTING WITH APACHE WEB SERVER
Please install Apache on all nodes

1.yum install httpd

– Create an index.html on DocumentRoot node1

1.vi /var/www/html/index.html

Fill with the following example

This is node1

Save and restart service Apache

1.service httpd restart

Please try to access node1 via browser. You will see a text This is node1
– Create an index.html on DocumentRoot node2

1.vi /var/www/html/index.html

Fill with the following example

This is node2

Save and restart service Apache

1.service httpd restart

Please try to access node2 via browser. You will see a text This is node2
Integrate Apache with Heartbeat
Please change file /etc/ha.d/haresources on all nodes

1.vi /etc/ha.d/haresources

so that like below :

node1.imanudin.net IPaddr::192.168.80.93/24/eth0:0 httpd

Stop service Apache and configure automatic off at boot on all nodes (Service Apache will be handled by Heartbeat)

1.service httpd stop

2.chkconfig httpd off

Please try to access an alias IP from browser. You will see a text This is node1. Please try to stop Heartbeat service on node1 and refresh browser. You will see a text This is node2 (all services handled by Heartbeat on node1 will be taken by node2). For failback, please start again Heartbeat service on node1 (all services handled by Heartbeat on node2 will be taken again by node1)
You could also experiment with other services for online failover such as Samba, MySQL, MariaDB etc. The Heartbeat application only configure failover/failback, not data synchronize. For configuration data synchronize, i will explain on the next article
Good luck and hopefully useful