Mirroring

From Site5Wiki

[edit] Mirroring a Site

So, you have a site that is very critical to you and want to have a backup plan in case there is either scheduled maintenance or unscheduled maintenance of the server you're on? Great, in this industry you can never have to many backup or "disaster recovery" plans. In this wiki we'll discuss how to setup a mirror of your site so that all you need to do is change DNSes and your site will be up. We'll also talk about possible issues and ways to overcome those.

[edit] What is a mirror?

Just like the real world mirror, a mirrored site is a duplication of the "real" site. However, unlike a real world mirror, a mirrored site can be switched too at any time and back again.

[edit] Why would I want a mirrored site?

Well, lets say your site is critical to you or you want to keep a backup site elsewhere so you can switch to it at any time. Having this in place ensures that your site will be live all the time for a good majority of people even during server maintenance.

[edit] What do I need?

Good question and here is where the important stuff comes in to play -- money.

1. For a mirrored site to work, you need at least:
2. Two hosting accounts on two different servers -- both servers can, of course, be with us.
3. External DNS to making switching fast -- you could use our DNS for both, however, you would need to contact us to have the switching done. Using the external DNS does make it more complex and hard to manage sometimes as you'll need to manually modify the zones there for new domains and new subdomains. As such, this approach is only recommended for those "die hards". However, it is the best way as changing the nameservers themselves at your registrar would cause DNS propagation delays. The external/secondary DNS should allow for fast TTLs.

   DNS Made Easy offers an automated switching system if it can't contact the primary server, however, this is a paid service.

4. Secondary/backup account should allow SSH access -- all accounts here are allowed SSH. This will be for the mirroring process.
5. SSH-keys -- this will allow for automated logging in to the backup/secondary account.
6. Exact user names and paths -- last but most important is to have accounts that have the exact username for the account and for the mysql users. Also, it's important to have the same path for the accounts on the server -- not an issue here usually.

[edit] Why are you providing this information?

As noted earlier, you can never have enough backups. A mirrored site is a backup or disaster recovery plan. Our servers are very reliable, however, maintenance is a fact of server life and for those that need their sites up all the time, a mirrored site is the best way to go.

[edit] Setup

Okay, so now that we know what mirroring is as defined in this wiki, we can move forward on setup. Setting up can be easy or complex, but it depends on what you want to do.

1. Sign-up for a new account here at Site5 -- it's easier to sign up here as paths and usernames will be the same. Otherwise you would need to signup else where that you can have the same user and home directory paths.
2. Setup the new account, lets call it secondary from now on, to match perfectly with the other "main" account.
   • You'll want to make sure everything matches from databases to mysql users to email accounts.
3. Setup SSH
   1. Shared/Multisite
      1. Go to SiteAdmin for the secondary account.
      2. Now go to Advanced Tools --> Get Shell Access and click Enable Shell Access.
      3. You should have shell access now.
   2. Reseller
      1. Login to WHM
      2. Go to Modify an Account --> and locate the desired account.
      3. Check "Shell Access" and save.
   3. Repeat for the primary account if needed.
   4. Generate keys for automatic login.
      1. Login to SSH on the "primary" account.
      2. Type in: ssh-keygen -t rsa
      3. Type in "~/.ssh/rsync" (without quotes) for the file name
      4. Press the Enter key for the passphrase -- DO NOT type in anything as it will require it during rsync and we want this to be automated.

         NOTE: no passphrase keys should only be used on systems that aren't important. DO NOT use the key for authentication on the primary account.

      5. Copy the public key file, rsync.pub, to the secondary account's authorized_keys file by doing the following command:

         scp ~/.ssh/rsync.pub user@domain.com:.ssh/authorized_keys

         Change text in bold to proper values.

      6. Test key by using the following command:

         ssh -i~/.ssh/rsync user@domain.com

4. Create a file called mirror.sh in the home directory:

   touch ~/mirror.sh

5. Edit mirror.sh:

   nano mirror.sh

6. Paste (right click) the following in to that file:

#!/bin/bash
### This script copies the contents of public_html and databases specifed to the mirror server
RemoteUser=""
Remotehost=""
SQLHost=""
RemoteSQLUser=""
RemoteSQLPass=""
SQLUser=""
SQLPass=""

dbToGet=(db1 db2 db3)
# Each database should be separated by a space, i.e. database1 db2 db3 db4

#### Lets start, shall we ####

rsync --archive --delete --verbose --compress -e "ssh -i ~/.ssh/rsync -p2200" ~/public_html $RemoteUser@$RemoteHost:~/ >> rsync.log

## Dump MySQL ##
for db in $dbToGet
do
# Get load so we're not stressing the server
while [ `uptime | cut -d, -f4 | cut -d\: -f2 | tr " " "\n" | tail -1 | cut -d\. -f1` -gt 5 ]
do
sleep 60
# Load is above 5, to keep stress down on the server lets wait until it's below 3 to process
# another database.
done

mysqldump -u$SQLUser -p$SQLPass --quote-names --allow-keywords --opt $db | mysql -h$SQLHost -u$RemoteSQLUser -p$RemoteSQLPass $db
done
#=- MySQL Backup Done -=

1. Create a cronjob that will run the above shell script.
   1. Go in to the primary account's SiteAdmin or CPanel and go to Cronjob.
   2. Put 07 in the minute and 02 in the hour fields -- that will run every 2:07 AM -- it's best to offset from other crons and odd numbers are usually better and do this type of work during off-peak hours.

      If you need more often runs, change the hours to be like */3 for every three hours, */6 for every six hours and so on.

   3. Put the following in the command field:

      nice +15 ~/mirror.sh

[edit] Notes

• Steps listed in this wiki presume that you have the same users and everything matches. You can get complex and do replaces as you change and what not. Maybe one day that will be in here.
• Steps listed here presume the rsync is ran from the primary account. You can run it on the secondary, but you would need to reverse some steps.

[edit] Other Methods and Tools

• HTTrack/WebHTTrack:

  HTTrack is an easy-to-use offline browser utility. It allows you to download a Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. Simply open a page of the mirrored Web site in your browser, and you can browse the site from link to link, as if you were viewing it online. HTTrack can also update an existing mirrored site, and resume interrupted downloads. WebHTTrack is a Web-based GUI for HTTrack. — Image:Freshmeat.png Freshmeat download page

  • Official site
  • KDE GUI front for HTTrack