How to sync the data between two servers without using SAN replication? Do you have better than RSYNC tool for this job ? I don’t think so, you we will not get better than RSYNC. It uses ‘rsync algorithm’ which provides a very fast method for syncing the directories or filesystems. An important feature of rsync is that the mirroring takes place with only one transmission in each direction and which is not available in other similar programs.
Rsync’s default port is 873 and it’s an opensource software. Rsync is available for Unix, Linux and windows operating systems. You can freely download rsync source code from rsync.samba web portal.
Here we will see how to sync data between two servers using automated script.
Operating system: Red Hat Linux
Source Server IP: 192.168.10.20 (mylinz1)
Source Server Path: /db/oracle/
Destination Server IP: 192.168.10.25 (mylinz2)
Destination Server Path: /db/oracle-bck/
Before proceeding to rsync part,you need to configure key-less authentication to ensure each can communicate using that.
Configuring key-less authentication
1. Verify whether your host will allow to perform RSA key-less authentication.If you didn’t get similar output,then you need to comment out the lines in sshd_config.
[root@mylinz1 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#"
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
[root@mylinz1 ~]#
[root@mylinz2 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#"
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
2. Generate the keygen if you didn’t have one already. Here the user is “root”.
[root@mylinz1 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
94:e6:6f:66:bb:cd:a8:30:4d:90:94:31:ae:64:f6:5e root@mylinz1
The key's randomart image is:
+--[ RSA 2048]----+
| +o |
| o.o . |
| + + + |
| + o = |
| . . E |
| . + . |
| + . = |
| o + = |
| ..+.o |
+-----------------+
[root@mylinz1 ~]# cd .ssh/
[root@mylinz1 .ssh]# ls -lrt
total 8
-rw-r--r--. 1 root root 394 Jun 19 00:43 id_rsa.pub
-rw-------. 1 root root 1671 Jun 19 00:43 id_rsa
[root@mylinz1 .ssh]#
[root@mylinz2 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
e7:a8:19:ac:dd:e3:28:b3:42:00:a0:84:4a:10:4e:fe root@mylinz2
The key's randomart image is:
+--[ RSA 2048]----+
|B+ |
|O. |
|=o |
|o . |
| . E S . |
| . . + |
| . o . . |
| . oo *. |
| .o+=.o. |
+-----------------+
[root@mylinz2 ~]# cd .ssh/
[root@mylinz2 .ssh]# ls -lrt
total 12
-rw-r--r--. 1 root root 395 Jun 19 00:17 known_hosts
-rw-------. 1 root root 1675 Jun 19 00:44 id_rsa
-rw-r--r--. 1 root root 394 Jun 19 00:44 id_rsa.pub
3. Share the “id_rsc.pub” file across the servers to enable the ssh key-less authentication.
[root@mylinz1 .ssh]# scp -r id_rsa.pub 192.168.10.25:/root/.ssh/authorized_keys
The authenticity of host '192.168.10.25 (192.168.10.25)' can't be established.
RSA key fingerprint is 5a:56:fd:69:cf:f2:b8:78:b9:67:e8:d0:f2:a4:ef:cb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.25' (RSA) to the list of known hosts.
root@192.168.10.25's password:
id_rsa.pub 100% 394 0.4KB/s 00:00
[root@mylinz1 .ssh]#
[root@mylinz2 .ssh]# scp -r id_rsa.pub 192.168.10.20:/root/.ssh/authorized_keys
root@192.168.10.20's password:
id_rsa.pub 100% 394 0.4KB/s 00:00
[root@mylinz2 .ssh]#
4. Verify your work.
[root@mylinz2 ~]# ssh 192.168.10.20
Last login: Wed Jun 19 00:42:27 2013 from 192.168.10.101
[root@mylinz1 ~]# ssh 192.168.10.25
Last login: Tue Jun 18 23:59:19 2013 from 192.168.10.101
[root@mylinz2 ~]#
Let’s move to RSYNC part.
RSYNC SCRIPT
Here is the rsync script which will be used for syncing the data between servers’ mylinz1 & mylinz2.
Here I am running script from mylinz1 to sync the data.
[root@mylinz1 ~]# cat rsync_oracle.sh
#!/bin/bash
# RSYNC SCRIPT TO SYNC TWO SERVER'S SPECIFIC DIRECTORIES
# Website:solariscat.blogspot.com
SOURCE_PATH='/db/oracle/'
SOURCE_SERVER='192.168.10.20' #Added for reference
DESTINATION_PATH='/db/oracle-bck/'
DESTINATION_HOST='192.168.10.25'
DESTINATION_USER='root'
LOGFILE='rsync_oralce.log'
echo $'\n\n' >> $LOGFILE
rsync -av --rsh=ssh $SOURCE_PATH $DESTINATION_USER@$DESTINATION_HOST:$DESTINATION_PATH 2>&1 >> $LOGFILE
echo "Sync Completed at:`/bin/date`" >> $LOGFILE
[root@mylinz1 ~]#pwd
/root
[root@mylinz1 ~]#chmod 700 rsync_oracle.sh
This script creates log as well with newly synchronized files information.
Testing rsync script:
1. Run the script manually.
[root@mylinz1 ~]#./rsync_oracle.sh
2. Verify the log file.
[root@mylinz1 ~]# tail -1rsync_oralce.log
kshrc
latrace.conf
ld.so.cache
ld.so.conf
libaudit.conf
libuser.conf
sent 160321 bytes received 1155 bytes 107650.67 bytes/sec
total size is 156728 speedup is 0.97
Completed at:Wed Jun 19 01:02:25 IST 2013
[root@mylinz1 oracle]#
3. Create a new file to sync the data to mylinz2 server.
[root@mylinz1 oracle]#cd /db/oracle/
[root@mylinz1 oracle]# touch verify_rsync
[root@mylinz1 oracle]# ls -lrt verify_rsync
-rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync
[root@mylinz1 oracle]# cd /root
[root@mylinz1 ~]# ./rsync_oracle.sh
[root@mylinz1 ~]#
[root@mylinz1 ~]# tail -10 rsync_oralce.log
sending incremental file list
./
verify_rsync
sent 1093 bytes received 34 bytes 2254.00 bytes/sec
total size is 156728 speedup is 139.07
Completed at:Wed Jun 19 01:05:03 IST 2013
[root@mylinz1 ~]#
4. Verify the whether the newly created file synced in mylinz2.
[root@mylinz2 ~]# cd /db/oracle-bck/
[root@mylinz2 oracle-bck]# ls -lrt verify_rsync
-rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync
[root@mylinz2 oracle-bck]#
That’s it. Our rsync script is working fine.
Automating sync
If you want to sync the data between two servers automatically on preferred time interval,you can add the script in to crontab.
Add the below line in root’s crontab to sync the data for every 5 minutes.
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /root/rsync.sh 2>&1 > /dev/null
That’s it!!!
