The zbackup.sh script is used to perform "hot" backups using native features of ZFS in conjunction with the command-line Veritas NetBackup utilities. ZFS allows us to take a live snapshot of any ZFS filesystems on the system. Because ZFS only records the changed blocks, it can complete a full snapshot at blazing speeds.
Once the snapshot is ready, we can use zfs send to pipe it through a fifo. In the meantime, zbackup.sh starts up another process to execute the NetBackup client. It reads in from the fifo and sends the backup stream to a policy on the NetBackup media server.
Zbackup relies on a very simple configuration file delimited by whitespace. Each entry describes a filesystem, its mount point, and whether zbackup.sh should perform a backup of that filesystem. In this example, we have four zfs mount points but we are only backing up our PostgreSQL data in /pgdata/main.
N zfsarray /zfsarray N zfsarray/crash /var/crash N zfsarray/homes /export/home Y zfsarray/pgdata /pgdata/main
We'll start off by reviewing the variables defined at the top of zbackup.sh and looking at the order in which our tasks are called. Then we'll dig deeper to see how each routine performs its functionality.
BASESNAP=lastfull INCREMENTAL=no BACKUPCONF=/etc/zbackup.conf BFILE=/var/log/pgods-backup.filelist BPCMD=/usr/openv/netbackup/bin/bpbackup BPLOG=/var/log/netbackup_backup.log BPCLASS=PGods BPSCHED=PGODS.UserBackup RUN=true INCDUMPDIR=/tmp
- BASESNAP - The suffix to append to our FULL backup snapshot. This can be seen using zfs list.
- INCREMENTAL - By default, we want to run a FULL backup. This can be overridden with the -i option to zbackup.sh.
- BACKUPCONF - Where to find our configuration file.
- BFILE - The lock file used by zbackup.sh. Zbackup will exit if this file exists. Force override with the -f flag.
- BPCMD - Path to the NetBackup command-line backup utility.
- BPLOG - Path to the NetBackup log file.
- BPCLASS - Name of our NetBackup policy, as defined on the NetBackup media server.
- BPSCHED - Name of our NetBackup policy schedule, as defined on the NetBackup media server.
- RUN - By overriding this value with the -n flag, Zbackup will tell us what it would have done, without actually performing the backup.
- INCDUMPDIR - Directory where we will create our fifo.
clear_full sanity postgres_start_backup snap postgres_stop_backup backup
Zbackup starts by performing a zfs destroy on any FULL snapshots we might be duplicating (FULL mode). It then runs a validation routine to confirm that our FULL snapshot was destroyed (FULL mode) or that we have a FULL snapshot available (INCREMENTAL mode).
Next, we perform the actual zfs snapshot to fifo. Once this is completed, we inform PostgreSQL that we are finished (pg_stop_backup()).
Finally, we start the NetBackup client to read from the fifo and begin the backup to tape.
This section is not meant to be an exhaustive dissection of all that can go wrong with zbackup.sh. Rather, it is intended to give the new user an introduction into using the tools behind Zbackup. If we can reproduce a problem manually, it provides us with better feedback and a better understanding of where the process flow breaks down.
First, you should become familiar with how some of the basic ZFS functionality works. At a very rudimentary level, zfs list will tell us which ZFS filesystems are available and in use.
$ zfs list NAME USED AVAIL REFER MOUNTPOINT intmirror 24.6G 42.3G 24.5K /intmirror intmirror/pgdata 24.6G 42.3G 12.9G /pgdata/main intmirror/pgdata@20080326 3.87G - 12.6G - intmirror/pgdata@20080327 4.09G - 12.8G - intmirror/pgdata@lastfull 3.75G - 12.7G - storedge_2 1.66T 757G 1.45M /storedge_2 storedge_2/pgdata 1.66T 757G 1.48T /pgdata/alldata1 storedge_2/pgdata@20080326 42.4G - 1.43T - storedge_2/pgdata@20080327 38.4G - 1.43T - storedge_2/pgdata@lastfull 53.5G - 1.48T -
Let's imagine that we're having difficulties with the FULL backups for intmirror/pgdata. We've already created a manual snapshot that we can use for testing. With this we can emulate zbackup.sh by issuing a zfs send to the fifo. If it doesn't already exist, you'll need to create the fifo.
# zfs send intmirror/pgdata@lastfull >> /tmp/intmirror\:pgdata.lastfull.full
If no problems were reported, we can proceed to running the NetBackup client. In a separate terminal, run the following (changing policy and schedule names were appropriate).
# /usr/openv/netbackup/bin/bpbackup -w -c PGdata -s PGDATA.UserBackup -L /var/log/netbackup_backup.log \ /tmp/intmirror\:pgdata.lastfull.full
If there are any problems with the policy configuration, you should see errors at this point. Otherwise, everything should be running smoothly and you should be able to monitor the backup job on the media server.