root/branches/omnipitr-with-slave-backups-calling-master/doc/omnipitr-backup-slave.pod

Revision 233, 10.9 kB (checked in by depesz, 3 years ago)

alpha version

Line 
1 =head1 OmniPITR - omnipitr-backup-slave
2
3 =head2 USAGE
4
5 /some/path/omnipitr/bin/omnipitr-backup-slave [options]
6
7 Options:
8
9 =over
10
11 =item --data-dir (-D)
12
13 Where PostgreSQL datadir is located (path)
14
15 =item --database (-d)
16
17 Which database to connect to to issue required SQL queries. Defaults to
18 template1.
19
20 This is used only when --call-master is given.
21
22 =item --host (-h)
23
24 Which host to connect to when connecting to database to backup. Shouldn't really
25 be changed in 99% of cases. Defaults to empty string - i.e. use UNIX sockets.
26
27 This is used only when --call-master is given.
28
29 =item --port (-P)
30
31 Which port to connect to when connecting to database. Defaults to 5432.
32
33 This is used only when --call-master is given.
34
35 =item --username (-U)
36
37 What username to use when connecting to database. Defaults to postgres.
38
39 This is used only when --call-master is given.
40
41 =item --source (-s)
42
43 Where are WAL files delivered by L<omnipitr-archive> and used by
44 L<omnipitr-restore>
45
46 You can also specify compression for source. Check L<COMPRESSION>
47 section of the doc.
48
49 =item --dst-local (-dl)
50
51 Where to copy the hot backup files on current server (you can provide many of
52 these).
53
54 You can also specify compression per-destination. Check L<COMPRESSION>
55 section of the doc.
56
57 =item --dst-remote (-dr)
58
59 Where to copy the hot backup files on remote server. Supported ways to transport
60 files are rsync and rsync over ssh. Please see L<DESCRIPTION> for more
61 information (you can provide many of these)
62
63 You can also specify compression per-destination. Check L<COMPRESSION>
64 section of the doc.
65
66 =item --temp-dir (-t)
67
68 Where to create temporary files (defaults to /tmp or I<$TMPDIR> environment
69 variable location)
70
71 =item --log (-l)
72
73 Name of logfile (actually template, as it supports %% L<strftime(3)>
74 markers. Unfortunately due to the %x usage by PostgreSQL, We cannot use %%
75 macros directly. Instead - any occurence of ^ character in log dir will be first
76 changed to %, and later on passed to strftime.
77
78 =item --filename-template (-f)
79
80 Template for naming output files. Check L<FILENAMES> section for details.
81
82 =item --pid-file
83
84 Name of file to use for pidfile. If it is specified, than only one copy of
85 I<omnipitr-backup-slave> (with this pidfile) can run at the same time.
86
87 Trying to run second copy of I<omnipitr-backup-slave> will result in an error.
88
89 =item --removal-pause-trigger (-p)
90
91 name of file to be created to pause removal of old/obsolete segments. Should be
92 the same as condigured removal-pause-trigger for L<omnipitr-restore>.
93
94 =item --call-master (-cm)
95
96 If this option is given, L<omnipitr-backup-slave> will issue
97
98     SELECT pg_start_backup( '...' );
99
100 and
101
102     SELECT pg_stop_backup()
103
104 on master database.
105
106 Backups on PostgreSQL 9.0+, on slave, without --call-master can fail when later
107 used.
108
109 =item --verbose (-v)
110
111 Log verbosely what is happening.
112
113 =item --not-nice (-nn)
114
115 Do not use nice for compressions.
116
117 =item --gzip-path (-gp)
118
119 Full path to gzip program - in case you can't set proper PATH environment
120 variable.
121
122 =item --bzip2-path (-bp)
123
124 Full path to bzip2 program - in case you can't set proper PATH environment
125 variable.
126
127 =item --lzma-path (-lp)
128
129 Full path to lzma program - in case you can't set proper PATH environment
130 variable.
131
132 =item --nice-path (-np)
133
134 Full path to nice program - in case you can't set proper PATH environment
135 variable.
136
137 =item --tar-path (-tp)
138
139 Full path to tar program - in case you can't set proper PATH environment
140 variable.
141
142 =item --psql-path (-pp)
143
144 Full path to psql program - in case you can't set proper PATH environment
145 variable.
146
147 =item --rsync-path (-rp)
148
149 Full path to rsync program - in case you can't set proper PATH environment
150 variable.
151
152 =item --pgcontroldata-path (-pp)
153
154 Full path to pg_controldata program - in case you can't set proper PATH
155 environment variable.
156
157 =back
158
159 =head2 DESCRIPTION
160
161 Running this program should be done by cronjob, or manually by database
162 administrator.
163
164 As a result of running it there are 2 files, usually named
165 HOST-data-YYYY-MM-DD.tar and HOST-xlog-YYYY-MM-DD.tar. These files can be
166 optionally compressed and delivered to many places - both local (on the same
167 server) or remote (via rsync).
168
169 Which options should be given depends only on installation, but generally you
170 will need at least:
171
172 =over
173
174 =item * --data-dir
175
176 Backup will process files in this directory.
177
178 =item * --log
179
180 to make sure that information is logged someplace about archiving progress
181
182 =item * one of --dst-local or --dst-remote
183
184 to specify where to send the backup files to
185
186 =back
187
188 Of course you can provide many --dst-local or many --dst-remote or many mix of
189 these.
190
191 Generally omnipitr-backup-slave will try to deliver WAL segment to all
192 destinations. In case remote destination will fail, omnipitr-backup-slave will
193 retry 3 times, with 5 minute delay between tries.
194
195 In case of errors when writing to local destination - it is skipped, and error
196 is logged.
197
198 Backups will be transferred to destinations in this order:
199
200 =over
201
202 =item 1. All B<local> destinations, in order provided in command line
203
204 =item 2. All B<remote> destinations, in order provided in command line
205
206 =back
207
208 =head3 Remote destination specification
209
210 I<omnipitr-backup-slave> delivers backup files to destination using rsync
211 program.  Both direct-rsync and rsync-over-ssh are supported (it's better to use
212 direct rsync - it uses less resources due to lack of encryption.
213
214 Destination url/location should be in a format that is usable by I<rsync>
215 program.
216
217 For example you can use:
218
219 =over
220
221 =item * rsync://user@remote_host/module/path/
222
223 =item * host:/path/
224
225 =back
226
227 To allow remote delivery you need to have rsync program. In case you're using
228 rsync over ssh, I<ssh> program has also to be available.
229
230 In case your rsync/ssh programs are in custom directories simply set I<$PATH>
231 environemnt variable before starting PostgreSQL.
232
233 =head2 COMPRESSION
234
235 Every destination (and source of xlogs) can have specified compression. To use
236 it you should prefix destination path/url with compression type followed by '='
237 sign.
238
239 Allowed compression types:
240
241 =over
242
243 =item * gzip
244
245 Compresses with gzip program, used file extension is .gz
246
247 =item * bzip2
248
249 Compresses with bzip2 program, used file extension is .bz2
250
251 =item * lzma
252
253 Compresses with lzma program, used file extension is .lzma
254
255 =back
256
257 If you want to pass any extra arguments to compression program, you can either:
258
259 =over
260
261 =item * make a wrapper
262
263 Write a program/script that will be named in the same way your actual
264 compression program is named, but adding some parameters to call
265
266 =item * use environment variables
267
268 All of supported compression programs use environment variables:
269
270 =over
271
272 =item * gzip - GZIP
273
274 =item * bzip2 - BZIP2
275
276 =item * lzma - XZ_OPT
277
278 =back
279
280 For details - please consult manual to your choosen compression tool.
281
282 =back
283
284 B<It is strongly suggest to use only 1 compression method for all destinations>
285
286 =head2 FILENAMES
287
288 Naming of files for backups might be important depending on deployment.
289
290 Generally, generated filenames are named using templates, with default template
291 being:
292
293     __HOSTNAME__-__FILETYPE__-^Y-^m-^d.tar__CEXT__
294
295 Within template (specified with --filename-template option) you can use
296 following markers:
297
298 =over
299
300 =item * __HOSTNAME__
301
302 Name of server backup is made on - as reported by L<hostname(1)> program.
303
304 =item * __FILETYPE__
305
306 It is actually required to have __FILETYPE__ - it specifies whether the file
307 contains data (data) or xlog segments (xlog)
308
309 =item * __CEXT__
310
311 Based on compression algorithm choosen for given delivery. Can be empty (no
312 compression), or contains dot (.) and literal extension associated with choosen
313 compression program.
314
315 =item * any ^? markers
316
317 like in L<strftime(3)> call, but ^ will be first changed to %.
318
319 =back
320
321 Filename template is evaluated at start, so any timestamp (^? markers) will
322 relate to date/time of beginning of backup process.
323
324 =head2 TABLESPACES
325
326 If I<omnipitr-backup-master> detects additional tablespaces, they will be
327 also compressed to generated tarball.
328
329 Since the full path to the tablespace directory is important, and should be preserved,
330 and normally tar doesn't let you store files which path starts with "/" (as it
331 would be dangerous), I<omnipitr-backup-master> uses the following approach:
332
333 all tablespaces will be stored in tar, and upon extraction they will be put in
334 the directory "tablespaces", and under it - there will be the full path to the tablespace
335 directory.
336
337 For example:
338
339 Assuming PostgreSQL PGDATA is in /var/lib/pgsql/data, and it has 3 extra
340 tablespaces placed in:
341
342 =over
343
344 =item * /mnt/san/tablespace
345
346 =item * /home/whatever/xxx
347
348 =item * /media/ssd
349
350 =back
351
352 generated DATA tarball will contain 2 directories:
353
354 =item * data - copy of /var/lib/pgsql/data
355
356 =item * tablespaces - which contains full directory structure leading to:
357
358 =over
359
360 =item * tablespaces/mnt/san/tablespace - copy of /mnt/san/tablespace
361
362 =item * tablespaces/home/whatever/xxx - copy of /home/whatever/xxx
363
364 =item * tablespaces/media/ssd - copy of /media/ssd
365
366 =back
367
368 =back
369
370 Thanks to this approach, if you'll create symlink "tablespaces" pointing to root
371 directory (I<ln -s / tablespaces>) B<before> exploding tarball - all tablespace
372 files will be created already in the correct places. This is of course not
373 necessary, but will help if you'd ever need to recover from such backup.
374
375 =head2 EXAMPLES
376
377 =head3 Minimal setup, with copying file to local directory:
378
379     /.../omnipitr-backup-slave -D /mnt/data -l /var/log/omnipitr/backup.log -dl /mnt/backups/
380
381 =head3 Minimal setup, with compression, and copying file to remote directory over rsync:
382
383     /.../omnipitr-backup-slave -D /mnt/data/ -l /var/log/omnipitr/backup.log -dr bzip2=rsync://slave/postgres/backups/
384
385 =head3 2 remote, compressed destinations, 1 local, with auto rotated logfile,
386 and modified filenames
387
388     /.../omnipitr-backup-slave -D /mnt/data/ -l /var/log/omnipitr/backup-^Y-^m-^d.log -dr bzip2=rsync://slave/postgres/backups/ -dr gzip=backups:/mnt/hotbackups/ -dl /mnt/backups/ -f "main-__FILETYPE__-^Y^m^d_^H^M^S.tar__CEXT__"
389
390 =head2 IMPORTANT NOTICES
391
392 =over
393
394 =item * If you're using compressed source dir (wal archive) -
395 I<omnipitr-backup-slave> has to uncompress all of xlogs before putting them in
396 .tar.XX. This means that you should have enough free disk space for this purpose
397 in the place where you create temporary directories. Alternarively - do not use
398 compression for sending WAL segments to standby server - in this case -
399 decompression will not be necessary.
400
401 =item * I<omnipitr-backup-slave> compresses whole source directory - i.e. all
402 files from there. This means that if you're using delay recovery (-w option to
403 I<omnipitr-restore>) - there will be more files there, and backup will be
404 larger. This is especially important if you're using compressed wal archive
405 (please see note above)
406
407 =item * If you're using L<omnipitr-backup-slave> on PostgreSQL 9.0+ you have to
408 use --call-master. Otherwise created backup might fail when used as base for
409 another replication slave that later on is promoted to standalone.
410
411 =back
412
413 =head2 COPYRIGHT
414
415 The OmniPITR project is Copyright (c) 2009-2010 OmniTI. All rights reserved.
416
Note: See TracBrowser for help on using the browser.