root/trunk/omnipitr/doc/omnipitr-backup-master.pod

Revision 216, 10.0 kB (checked in by depesz, 3 years ago)

fix language

Line 
1 =head1 OmniPITR - omnipitr-backup-master
2
3 =head2 USAGE
4
5 /some/path/omnipitr/bin/omnipitr-backup-master [options]
6
7 Options:
8
9 =over
10
11 =item --data-dir (-D)
12
13 Where PostgreSQL datadir is located (path)
14
15 =item --database (-d)
16
17 Which database to connect to to issue required SQL queries. Defaults to
18 template1.
19
20 =item --host (-h)
21
22 Which host to connect to when connecting to database to backup. Shouldn't really
23 be changed in 99% of cases. Defaults to empty string - i.e. use UNIX sockets.
24
25 =item --port (-p)
26
27 Which port to connect to when connecting to database. Defaults to 5432.
28
29 =item --username (-U)
30
31 What username to use when connecting to database. Defaults to postgres.
32
33 =item --xlogs (-x)
34
35 Directory that will be created. Then - will be used as source of xlogs to
36 archive. Afterwards - it will be removed. This directory should be the same as
37 --dst-backup for omnipitr-archive, and it shouldn't exist before running
38 I<omnipitr-backup-master>.
39
40 =item --dst-local (-dl)
41
42 Where to copy the hot backup files on current server (you can provide many of
43 these).
44
45 You can also specify compression per-destination. Check L<COMPRESSION>
46 section of the doc.
47
48 =item --dst-remote (-dr)
49
50 Where to copy the hot backup files on remote server. Supported ways to transport
51 files are rsync and rsync over ssh. Please see L<DESCRIPTION> for more
52 information (you can provide many of these)
53
54 You can also specify compression per-destination. Check L<COMPRESSION>
55 section of the doc.
56
57 =item --temp-dir (-t)
58
59 Where to create temporary files (defaults to /tmp or I<$TMPDIR> environment
60 variable location)
61
62 =item --state-dir (-s)
63
64 Name of directory to use as state-directory to handle errors when sending wal
65 segments to many locations.
66
67 =item --log (-l)
68
69 Name of logfile (actually template, as it supports %% L<strftime(3)>
70 markers. Unfortunately due to the %x usage by PostgreSQL, We cannot use %%
71 macros directly. Instead - any occurence of ^ character in log dir will be first
72 changed to %, and later on passed to strftime.
73
74 =item --filename-template (-f)
75
76 Template for naming output files. Check L<FILENAMES> section for details.
77
78 =item --pid-file
79
80 Name of file to use for pidfile. If it is specified, than only one copy of
81 I<omnipitr-backup-master> (with this pidfile) can run at the same time.
82
83 Trying to run second copy of I<omnipitr-backup-master> will result in an error.
84
85 =item --verbose (-v)
86
87 Log verbosely what is happening.
88
89 =item --not-nice (-nn)
90
91 Do not use nice for compressions.
92
93 =item --gzip-path (-gp)
94
95 Full path to gzip program - in case you can't set proper PATH environment
96 variable.
97
98 =item --bzip2-path (-bp)
99
100 Full path to bzip2 program - in case you can't set proper PATH environment
101 variable.
102
103 =item --lzma-path (-lp)
104
105 Full path to lzma program - in case you can't set proper PATH environment
106 variable.
107
108 =item --nice-path (-np)
109
110 Full path to nice program - in case you can't set proper PATH environment
111 variable.
112
113 =item --tar-path (-tp)
114
115 Full path to tar program - in case you can't set proper PATH environment
116 variable.
117
118 =item --rsync-path (-rp)
119
120 Full path to rsync program - in case you can't set proper PATH environment
121 variable.
122
123 =item --psql-path (-pp)
124
125 Full path to psql program - in case you can't set proper PATH environment
126 variable.
127
128 =back
129
130 =head2 DESCRIPTION
131
132 Running this program should be done by cronjob, or manually by database
133 administrator.
134
135 As a result of running it there are 2 files, usually named
136 HOST-data-YYYY-MM-DD.tar and HOST-xlog-YYYY-MM-DD.tar. These files can be
137 optionally compressed and delivered to many places - both local (on the same
138 server) or remote (via rsync).
139
140 Which options should be given depends only on installation, but generally you
141 will need at least:
142
143 =over
144
145 =item * --data-dir
146
147 Backup will process files in this directory.
148
149 =item * --log
150
151 to make sure that information is logged someplace about archiving progress
152
153 =item * one of --dst-local or --dst-remote
154
155 to specify where to send the backup files to
156
157 =back
158
159 Of course you can provide many --dst-local or many --dst-remote or many mix of
160 these.
161
162 Generally omnipitr-backup-master will try to deliver WAL segment to all
163 destinations. In case remote destination will fail, omnipitr-backup-master will
164 retry 3 times, with 5 minute delay between tries.
165
166 In case of errors when writing to local destination - it is skipped, and error
167 is logged.
168
169 Backups will be transferred to destinations in this order:
170
171 =over
172
173 =item 1. All B<local> destinations, in order provided in command line
174
175 =item 2. All B<remote> destinations, in order provided in command line
176
177 =back
178
179 =head3 Remote destination specification
180
181 I<omnipitr-backup-master> delivers backup files to destination using rsync
182 program.  Both direct-rsync and rsync-over-ssh are supported (it's better to use
183 direct rsync - it uses less resources due to lack of encryption.
184
185 Destination url/location should be in a format that is usable by I<rsync>
186 program.
187
188 For example you can use:
189
190 =over
191
192 =item * rsync://user@remote_host/module/path/
193
194 =item * host:/path/
195
196 =back
197
198 To allow remote delivery you need to have rsync program. In case you're using
199 rsync over ssh, I<ssh> program has also to be available.
200
201 In case your rsync/ssh programs are in custom directories simply set I<$PATH>
202 environemnt variable before starting PostgreSQL.
203
204 =head2 COMPRESSION
205
206 Every destination can have specified compression. To use it you should prefix
207 destination path/url with compression type followed by '=' sign.
208
209 Allowed compression types:
210
211 =over
212
213 =item * gzip
214
215 Compresses with gzip program, used file extension is .gz
216
217 =item * bzip2
218
219 Compresses with bzip2 program, used file extension is .bz2
220
221 =item * lzma
222
223 Compresses with lzma program, used file extension is .lzma
224
225 =back
226
227 All compressions are done I<on B<NICE>> to make the operation as unobtrusive as
228 possible.
229
230 If you want to pass any extra arguments to compression program, you can either:
231
232 =over
233
234 =item * make a wrapper
235
236 Write a program/script that will be named in the same way your actual
237 compression program is named, but adding some parameters to call
238
239 =item * use environment variables
240
241 All of supported compression programs use environment variables:
242
243 =over
244
245 =item * gzip - GZIP
246
247 =item * bzip2 - BZIP2
248
249 =item * lzma - XZ_OPT
250
251 =back
252
253 For details - please consult manual to your choosen compression tool.
254
255 =back
256
257 B<It is strongly suggest to use only 1 compression method for all destinations>
258
259 =head2 FILENAMES
260
261 Naming of files for backups might be important depending on deployment.
262
263 Generally, generated filenames are named using templates, with default template
264 being:
265
266     __HOSTNAME__-__FILETYPE__-^Y-^m-^d.tar__CEXT__
267
268 Within template (specified with --filename-template option) you can use
269 following markers:
270
271 =over
272
273 =item * __HOSTNAME__
274
275 Name of server backup is made on - as reported by L<hostname(1)> program.
276
277 =item * __FILETYPE__
278
279 It is actually required to have __FILETYPE__ - it specifies whether the file
280 contains data (data) or xlog segments (xlog)
281
282 =item * __CEXT__
283
284 Based on compression algorithm choosen for given delivery. Can be empty (no
285 compression), or contains dot (.) and literal extension associated with choosen
286 compression program.
287
288 =item * any ^? markers
289
290 like in L<strftime(3)> call, but ^ will be changed to % first.
291
292 =back
293
294 Filename template is evaluated at start, so any timestamp (^? markers) will
295 relate to date/time of beginning of backup process.
296
297 =head2 TABLESPACES
298
299 If I<omnipitr-backup-master> detects additional tablespaces, they will be
300 also compressed to generated tarball.
301
302 Since the full path to the tablespace directory is important, and should be preserved,
303 and normally tar doesn't let you store files which path starts with "/" (as it
304 would be dangerous), I<omnipitr-backup-master> uses the following approach:
305
306 all tablespaces will be stored in tar, and upon extraction they will be put in
307 the directory "tablespaces", and under it - there will be the full path to the tablespace
308 directory.
309
310 For example:
311
312 Assuming PostgreSQL PGDATA is in /var/lib/pgsql/data, and it has 3 extra
313 tablespaces placed in:
314
315 =over
316
317 =item * /mnt/san/tablespace
318
319 =item * /home/whatever/xxx
320
321 =item * /media/ssd
322
323 =back
324
325 generated DATA tarball will contain 2 directories:
326
327 =item * data - copy of /var/lib/pgsql/data
328
329 =item * tablespaces - which contains full directory structure leading to:
330
331 =over
332
333 =item * tablespaces/mnt/san/tablespace - copy of /mnt/san/tablespace
334
335 =item * tablespaces/home/whatever/xxx - copy of /home/whatever/xxx
336
337 =item * tablespaces/media/ssd - copy of /media/ssd
338
339 =back
340
341 =back
342
343 Thanks to this approach, if you'll create symlink "tablespaces" pointing to root
344 directory (I<ln -s / tablespaces>) B<before> exploding tarball - all tablespace
345 files will be created already in the correct places. This is of course not
346 necessary, but will help if you'd ever need to recover from such backup.
347
348 =head2 EXAMPLES
349
350 =head3 Minimal setup, with copying file to local directory:
351
352     /.../omnipitr-backup-master -D /mnt/data -l /var/log/omnipitr/backup.log -dl /mnt/backups/
353
354 =head3 Minimal setup, with compression, and copying file to remote directory over rsync:
355
356     /.../omnipitr-backup-master -D /mnt/data/ -l /var/log/omnipitr/backup.log -dr bzip2=rsync://slave/postgres/backups/
357
358 =head3 2 remote, compressed destinations, 1 local, with auto rotated logfile,
359 and modified filenames
360
361     /.../omnipitr-backup-master -D /mnt/data/ -l /var/log/omnipitr/backup-^Y-^m-^d.log -dr bzip2=rsync://slave/postgres/backups/ -dr gzip=backups:/mnt/hotbackups/ -dl /mnt/backups/ -f "main-__FILETYPE__-^Y^m^d_^H^M^S.tar__CEXT__"
362
363 =head2 IMPORTANT NOTICES
364
365 =over
366
367 =item * It I<omnipitr-backup-master> will get stopped in B<hard> way (kill -9,
368 or multiple ctrl-c - it is possible that it will leave database with backup mode
369 still enabled. So, to prevent future problems with it, if you had to B<hard>
370 stop it - remember to connect to postgresql and issue:
371
372     SELECT pg_stop_backup();
373
374 =item * I<omnipitr-backup-master> should be running from the same system account
375 as PostgreSQL. It is technically possible to make it work with running from
376 another account, but it will definitely be more complicated and error-prone.
377
378 =back
379
380 =head2 COPYRIGHT
381
382 The OmniPITR project is Copyright (c) 2009-2010 OmniTI. All rights reserved.
383
Note: See TracBrowser for help on using the browser.