Reducing load caused by Rsync

By ayates

We're currently utilising rsync to take daily snapshots of our servers and backup to remote locations, lately this has been trigger high load notifications quite frequently. In order to reduce the load caused by these scripts I have gone with three small changes:

Rsync --whole-file

Rsync by default on local copies will just copy the whole file, but to reduce network traffic remote syncs are diff'd which greatly increases the CPU load. In our case we have our servers connected to each other on a gigabit port so network transfers are going to be fast, so we can force whole-file transfers and reduce the CPU load e.g.

rsync -av --whole-file -e ssh {source} {destination}

Weaker SSH Cipher

We can further reduce CPU load and increase the transfer rate on the remote transfers by using a weaker SSH encryption cipher. After some reading around and checking out benchmarks I decided to go with 'arcfour'

rsync -av --whole-file -e ssh -c arcfour {source} {destination}

Lower execution priority

FInally, I change the execution priority of the cronjobs so they run on spare processing power and should never take priority over processes - the backups are important but we can't have them causing delays in service. To do this we use 'nice' e.g.

nice -n 19 {command here}

So now my cronjobs look like this:

0 6 * * *   /usr/bin/nice -n 19 /root/scripts/backup.sh > /dev/null 2>&1