Snapshot backups using rsync - Revision history

Andrew: /* Example 3: Backup of C:\Documents and Settings on alaska */

2025-10-22T14:44:43Z

‎Example 3: Backup of C:\Documents and Settings on alaska

Admin: 4 revision(s)

2007-11-03T18:03:05Z

4 revision(s)

205.228.73.12 at 08:54, 9 August 2007

2007-08-09T08:54:16Z

205.228.74.12: /* Notes */

2006-09-26T11:00:29Z

‎Notes

205.228.73.13: /* Defining areas to be backed up */

2006-09-26T10:40:22Z

‎Defining areas to be backed up

205.228.74.12 at 10:33, 26 September 2006

2006-09-26T10:33:49Z

New page

== Intro ==
The scripts on this page are designed to allow the backup of a set of directories to a defined location, with the backups preserving the state of those directories from a number of different times in the past. They require a linux box with a free disk partition to act as the backup server, and the instructions below are designed for Ubuntu, so other distros may require slightly different steps. In summary, they will give you a snapshot directory with contents would look something like (as of 26th Sept 2006):
<pre>
Contents of /mnt/snapshot:
drwxr-xr-x 7 root root 4096 2006-09-26 00:01 daily.0
drwxr-xr-x 7 root root 4096 2006-09-25 00:00 daily.1
drwxr-xr-x 7 root root 4096 2006-09-24 00:00 daily.2
drwxr-xr-x 7 root root 4096 2006-09-23 00:00 daily.3
drwxr-xr-x 7 root root 4096 2006-09-22 00:00 weekly.0
drwxr-xr-x 5 root root 4096 2006-09-15 00:00 weekly.1
drwxr-xr-x 2 root root 4096 2006-09-08 00:00 weekly.2
drwxr-xr-x 2 root root 4096 2006-09-01 00:00 monthly.0
drwxr-xr-x 2 root root 4096 2006-08-01 00:00 monthly.1
drwxr-xr-x 2 root root 4096 2006-01-01 00:00 annual.0
</pre>
Note the timestamps of these directories - each one captures the state of the location(s) to be backed up at the time shown.
The actual backup process uses [http://samba.anu.edu.au/rsync/ rsync] and hard links to ensure that if a file doesn't change between two snapshots, then it doesn't take up any additional space (well, virtually no more space).
The contents of this page are based on Mike Rubel's instructions at: http://www.mikerubel.org/computers/rsync_snapshots/ - if you find difficulty understanding/using the scripts here, then you may wish to look at his page for additional help.

== Prerequisites ==
* Install '''rsync''' and '''nfs-kernel-server''' with apt-get (eg. ''sudo apt-get install rsync'')
* http://bretts.org/files/snapshot.tar.bz2

== Instructions ==
* Unzip the contents of '''snapshot.tar.bz2''' to a temporary directory.
* As root, copy the files and directories from the temporary directory to the implied filesystem location (eg. snapshot/etc/cron.d/snapshot -> /etc/cron.d/snapshot, snapshot/var/log/snapshot -> /var/log/snapshot etc.)
* As root, create the directories '''/root/.snapshot''' and '''/mnt/snapshot'''
* Add the following lines to '''/etc/fstab''', substituting your free disk partition, and it's filesystem type, for '''/dev/hdb1''' and '''ext3''':
<pre>
/dev/hdb1 /root/.snapshot ext3 ro 0 2
localhost:/root/.snapshot /mnt/snapshot nfs ro 0 2
</pre>
: (This mounts your backup partition as /root/.snapshot, away from prying eyes, and mounts the NFS exported version of it as /mnt/snapshot. We mount it twice so that non-root users NEVER see a writable version of the backup directory - see Mike Rubels page for more details.)
* Add the following line to '''/etc/exports''' (or create '''/etc/exports''' with this line if it doesn't already exist):
<pre>
/root/.snapshot 127.0.0.1(secure,sync,ro,no_root_squash)
</pre>
: (This defines the NFS share that we will mount as '''/mnt/snapshot''')
* Restart nfs-kernel-server (using ''sudo /etc/init.d/nfs-kernel-server restart'').

== Defining areas to be backed up ==
* For each directory tree to be backed up, create a directory in '''/etc/snapshot''' with a logical name for the backup (the name doesn't matter, as long as it's unique).
* Within the newly created directory, created a file called '''location''' (eg. '''/etc/snapshot/my-backup/location'''), the contents of which should be a single line defining the head of the directory tree to be backed up. This line may be in one of 3 formats:
** For a local directory, simply the fully qualified path to the location to back up (eg. '''/home/fred/docs''')
** For a remote directory accessed via an rsync daemon running on the remote machine, the location in the format ''machinename''::''rsync_module''/''subdirectory'' (eg. '''xp-box::c/Documents and Settings''')
** For a remote directory accessed via an ssh server running on the remote machine, the location in the format ''machinename'':''path'' (eg. '''other-host:/usr/local/src''')
: For remote hosts, an rsync daemon is generally preferable, both for performance reasons, and for ease of use. If ssh is to be used, ssh keys will need to be defined to stop the scripts prompting for a password whenever they're run. However, if the data is being transferred between hosts across an insecure network, ssh may be needed for reasons of security (rsync transmits data between hosts unencrypted, and has a simplistic security model).
* Also within the directory, create a file called '''filters''' (eg. '''/etc/snapshot/my-backup/filters'''), which may contain multiple lines prefixed with a '''-''' to specify locations to exclude from the backup. This file is simply in rsync filters file format (see the rsync man page for more details).

Each one of the locations specified within '''/etc/snapshot''' will end up as a subdirectory of '''/mnt/snapshot''' (eg. this morning's backup of /home/fred will end up in '''/mnt/snapshot/daily.0/home/fred''').

=== Example 1: Backup of /home ===
'''/etc/snapshot/home/location''':
<pre>
/home
</pre>
'''/etc/snapshot/home/filters''':
<pre>
- /andrew/Download/
- /andrew/svn/
</pre>

=== Example 2: Backup of /etc ===
'''/etc/snapshot/etc/location''':
<pre>
/etc
</pre>
'''/etc/snapshot/etc/filters''':
<pre>
</pre>
Note that filters file is empty, since no subdirectories of /etc are excluded from the backup.

=== Example 3: Backup of C:\Documents and Settings on alaska ===
Note that alaska is an XP box running the cygwin rsync daemon.<br>
'''/etc/snapshot/alaska-home/location''':
<pre>
alaska::c/Documents and Settings
</pre>
'''/etc/snapshot/alaska-home/filters''':
<pre>
- /*/Local Settings/Temp
- /*/Local Settings/Temporary Internet Files
- /Andrew/My Documents/Downloads
</pre>

== Notes ==
* The partition used for /root/.snapshot should ideally be on a different disk to the files being backed up, to reduce the risk of a disk failure destroying both the data and the backups.
* The scripts can be used without defining /root/.snapshot as a separate partition, subject to the caveat about separate disks directly above. To do this, simply:
** Don't add the first of the two lines above to '''/etc/fstab'''
** In each of the snapshot scripts within '''/usr/local/sbin''' remove the calls near the beginning and end of the file to remount the /root/.snapshot partition (the beginning of each script remounts this partition as read/write, and the end remounts it as read).

@@ Line 92: / Line 92: @@
 ** Don't add the first of the two lines above to '''/etc/fstab'''
 ** In each of the snapshot scripts within '''/usr/local/sbin''' remove the calls near the beginning and end of the file to remount the /root/.snapshot partition (the beginning of each script remounts this partition as read/write, and the end remounts it as read).
 == Future Development ==
 * It would be nice to have the latest backup burnt to a DVD-RW which could be left in a dedicated DVD-RW drive attached to the backup server. This would help both because DVDs are more robust than hard drives in certain situations (eg. mechanical shock) in case of a problem (eg. power surge) which for some reason destroyed multiple drives on the machine. In addition, the DVD disk could be swapped out periodically to provide an off-site backup copy.
 * Some companies provide free or low cost disk space online - this could be used to store a copy of the latest backup for even greater robustness. In addition, since the process uses rsync, unless large numbers of backed up files were changing on a daily basis, there would be relatively little network traffic once the initial backup was rsynced across to the remote site.

← Older revision		Revision as of 11:00, 26 September 2006
Line 92:		Line 92:
	** Don't add the first of the two lines above to '''/etc/fstab'''		** Don't add the first of the two lines above to '''/etc/fstab'''
	** In each of the snapshot scripts within '''/usr/local/sbin''' remove the calls near the beginning and end of the file to remount the /root/.snapshot partition (the beginning of each script remounts this partition as read/write, and the end remounts it as read).		** In each of the snapshot scripts within '''/usr/local/sbin''' remove the calls near the beginning and end of the file to remount the /root/.snapshot partition (the beginning of each script remounts this partition as read/write, and the end remounts it as read).
		+
		+	== Future Development ==
		+	* It would be nice to have the latest backup burnt to a DVD-RW which could be left in a dedicated DVD-RW drive attached to the backup server. This would help both because DVDs are more robust than hard drives in certain situations (eg. mechanical shock) in case of a problem (eg. power surge) which for some reason destroyed multiple drives on the machine. In addition, the DVD disk could be swapped out periodically to provide an off-site backup copy.
		+	* Some companies provide free or low cost disk space online - this could be used to store a copy of the latest backup for even greater robustness. In addition, since the process uses rsync, unless large numbers of backed up files were changing on a daily basis, there would be relatively little network traffic once the initial backup was rsynced across to the remote site.

@@ Line 72: / Line 72: @@
 === Example 3: Backup of C:\Documents and Settings on alaska ===
-Note that alaska is an XP box running the cygwin rsync daemon.<br>
+Note that alaska is an XP box running an [[Windows_10_Tips#Running_rsyncd_automatically|rsync daemon]].<br>
 '''/etc/snapshot/alaska-home/location''':
 <pre>

← Older revision	Revision as of 18:03, 3 November 2007
(No difference)

@@ Line 83: / Line 83: @@
 - /Andrew/My Documents/Downloads
 </pre>
 == Notes ==