Table of Contents Previous Next


5 Operation : 5.2 Managing Backups Using a Retention Policy

1.
Determine and set a retention policy in the BART configuration file. A retention policy is a rule that determines when a backup is considered obsolete. The retention policy can be applied globally to all servers (see Section 4.1), but each server can override the global retention policy with its own (see Section 4.2.5).
2.
Use the MANAGE subcommand to categorize and manage backups according to the retention policy. Such functionality includes determining which active backups should be considered obsolete at this current point in time, selecting backups to keep indefinitely, and physically deleting obsolete backups. When an obsolete backup is deleted, its base backup taken with the BACKUP subcommand along with that backup’s archived WAL files are deleted.
3.
Once the retention policies have been determined and verified, you can create a cron job to periodically run the MANAGE subcommand to evaluate the backups and then list and/or delete the obsolete backups.
Section 5.2.1 provides an overview of the terminology and types of retention policies.
Section 5.2.2 describes the concept of marking the status of backups according to the retention policy.
Section 5.2.3 describes setting the different types of retention policies.
Section 5.2.4 describes the process of managing the backups such as marking the backup status, keeping selected backups indefinitely, listing obsolete backups, and deleting obsolete backups.
5.2.1 Overview
Active. The backup satisfies the retention policy applicable to its server. Such backups would be considered necessary to ensure the recovery safety for the server and thus should be retained.
Obsolete. The backup does not satisfy the retention policy applicable to its server. The backup is no longer considered necessary for the recovery safety of the server and thus can be deleted.
Keep. The backup is to be retained regardless of the retention policy applicable to its server. The backup is considered vital to the recovery safety for the server and thus should not be deleted for an indefinite period of time.
Redundancy Retention Policy. The redundancy retention policy relies on a specified, maximum number of most recent backups to retain for a given server. When the number of backups exceeds that maximum number, the oldest backups are considered obsolete (except for backups marked as keep). Section 5.2.3.1 describes this type of retention policy.
Recovery Window Retention Policy. The recovery window retention policy relies on a time frame (the recovery window) for when a backup should be considered active. The boundaries defining the recovery window are the current date/time (the ending boundary of the recovery window) and the date/time going back in the past for a specified length of time (the starting boundary of the recovery window). If the date/time the backup was taken is within the recovery window (that is, the backup date/time is on or after the starting date/time of the recovery window, then the backup is considered active, otherwise it is considered obsolete (except for backups marked as keep). Section 5.2.3.2 describes this type of retention policy.
Thus, for the recovery window retention policy, the recovery window time frame dynamically shifts so the end of the recovery window is always the current date/time when the MANAGE subcommand is run. As you run the MANAGE subcommand at future points in time, the starting boundary of the recovery window moves forward in time.
You can see the starting boundary of the recovery window at any point in time by running the SHOW-SERVERS subcommand. The RETENTION POLICY field of the SHOW-SERVERS subcommand displays the starting boundary of the recovery window.
When a backup is initially created with the BACKUP subcommand, it is always recorded with active status.
It is only by using the MANAGE subcommand at a particular point in time that active backups are evaluated to determine if their status should be changed to obsolete in accordance with the retention policy.
In addition, it is only when the MANAGE subcommand is invoked either with no options or with only the -s option (in order to specify the database server) that active backups are evaluated and also marked (that is, internally recorded by BART) as obsolete. See Section 5.4.6 for information on the MANAGE subcommand usage with its options.
Using the MANAGE subcommand, specify the -c option along with the backup identifier or name with the -i option. If you wish to keep this particular backup indefinitely, use -c keep, otherwise use -c nokeep.
If you used the -c nokeep option, the backup status is changed back to active. The next time the MANAGE subcommand is used, the backup is re-evaluated to determine if its status needs to be changed back to obsolete based upon the current retention policy in the BART configuration file.
Note that if the retention_policy parameter is set in a certain manner, you run the MANAGE subcommand to mark the backups according to that retention_policy setting, and then you change or disable the retention_policy parameter by commenting it out, the current, marked status of the backups are probably inconsistent with the current retention_policy setting.
If you wish to modify the backup status to be consistent with the current retention_policy setting, then perform the following:
If there are backups currently marked as obsolete that would no longer be considered obsolete under a new retention policy, run the MANAGE subcommand with the -c nokeep option for such backups to change the obsolete status to active status. You can also specify the -i all option, which would change all backups back to active status, including those currently marked as keep.
Run the MANAGE subcommand either with no options or with only the -s option to reset the marked status based upon the new retention_policy setting in the BART configuration file.
Deletion of backups with the MANAGE -d option relies on the last occasion when the backups have been marked.
The retention policy is determined by the retention_policy parameter in the BART configuration file. To set it globally for all servers, see Section 4.1. To set it by database server, see Section 4.2.5.
To use the redundancy retention policy, set retention_policy = max_number BACKUPS where max_number is a positive integer designating the maximum number of most recent backups.
The keyword BACKUPS must always be specified in plural form (for example, 1 BACKUPS).
BART will accept a maximum integer value of 2,147,483,647 for max_number, however, a realistic, practical value based on your system environment must always be used.
The redundancy retention policy is the default type of retention policy if all keywords BACKUPS, DAYS, WEEKS, and MONTHS following the max_number integer are omitted as shown by the following example:
The SHOW-SERVERS subcommand displays the 3 backup redundancy retention policy in the RETENTION POLICY field:
To use the recovery window retention policy, set the retention_policy parameter to the desired length of time for the recovery window in one of the following ways:
Set to max_number DAYS to define the start date/time recovery window boundary as the number of days specified by max_number going back in time from the current date/time.
Set to max_number WEEKS to define the start date/time recovery window boundary as the number of weeks specified by max_number going back in time from the current date/time.
Set to max_number MONTHS to define the start date/time recovery window boundary as the number of months specified by max_number going back in time from the current date/time.
The keywords DAYS, WEEKS, and MONTHS must always be specified in plural form (for example, 1 DAYS, 1 WEEKS, or 1 MONTHS).
BART will accept a maximum integer value of 2,147,483,647 for max_number, however, a realistic, practical value based on your system environment must always be used.
Invoking BART in debug mode (with the -d option) using the MANAGE subcommand with the -n option displays the calculated time length of the recovery window based on the retention_policy setting and the current date/time.
For example, using the following retention_policy settings:
If the MANAGE subcommand is invoked in debug mode along with the -n option on 2015-04-17, the following results are displayed:
For server ppas94, 72 hours translates to a recovery window of 3 days.
For server ppas94_2, 504 hours translates to a recovery window of 21 days (3 weeks).
For server pg94, 2160 hours translates to a recovery window of 90 days (3 months).
Note: For a setting of max_number MONTHS, the calculated, total number of days for the recovery window is dependent upon the actual number of days in the preceding months from the current date/time. Thus, max_number MONTHS is not always exactly equivalent to max_number x 30 DAYS. (For example, if the current date/time is in the month of March, a 1-month recovery window would be equivalent to only 28 days because the preceding month is February. Thus, for a current date of March 31, a 1-month recovery window would start on March 3.) However, the typical result is that the day of the month of the starting recovery window boundary will be the same day of the month of when the MANAGE subcommand is invoked.
Using the SHOW-SERVERS subcommand, the RETENTION POLICY field displays the start of the recovery window.
The start of the 3-day recovery window displayed in the RETENTION POLICY field is 2015-04-07 14:57:36 EDT when the SHOW-SERVERS subcommand is invoked on 2015-04-10.
At this current point in time, backups taken on or after 2015-04-07 14:57:36 EDT would be considered active. Backups taken prior to 2015-04-07 14:57:36 EDT would be considered obsolete except for backups marked as keep.
The start of the 3-week recovery window displayed in the RETENTION POLICY field is 2015-03-20 14:59:42 EDT when the SHOW-SERVERS subcommand is invoked on 2015-04-10.
At this current point in time, backups taken on or after 2015-03-20 14:59:42 EDT would be considered active. Backups taken prior to 2015-03-20 14:59:42 EDT would be considered obsolete except for backups marked as keep.
The start of the 3-month recovery window displayed in the RETENTION POLICY field is 2015-01-10 14:04:23 EST when the SHOW-SERVERS subcommand is invoked on 2015-04-10.
At this current point in time, backups taken on or after 2015-01-10 14:04:23 EST would be considered active. Backups taken prior to 2015-01-10 14:04:23 EST would be considered obsolete except for backups marked as keep.
The MANAGE subcommand is used to evaluate and categorize backups according to the retention policy set in the BART configuration file. When a backup is first created with the BACKUP subcommand, it always marked as active. Upon usage of the MANAGE subcommand, an active backup may be marked as obsolete. Obsolete backups can then be deleted.
Section 5.2.4.1 discusses the rules for deleting backups dependent upon the backup status and the subcommand used.
Section 5.2.4.2 shows how to retain a backup indefinitely by marking it as keep as well as resetting backups marked as obsolete and keep back to active status.
Section 5.2.4.3 demonstrates the general process for evaluating, marking, then deleting obsolete backups.
The MANAGE subcommand deletion relies solely upon how a backup status is currently marked (that is, internally recorded by BART). The current setting of the retention_policy parameter in the BART configuration file is ignored.
The DELETE subcommand relies solely upon the current setting of the retention_policy parameter in the BART configuration file. The current active, obsolete, or keep status of a backup is ignored.
Thus, the deletion behavior of the MANAGE subcommand and the DELETE subcommand are based on different aspects of the retention policy.
The specific deletion rules for the MANAGE and DELETE subcommands are as follows:
The MANAGE subcommand with the -d option can only delete backups marked as obsolete. This deletion occurs regardless of the current retention_policy setting in the BART configuration file.
Under a redundancy retention policy currently set with the retention_policy parameter in the BART configuration file, any backup regardless of its marked status, can be deleted with the DELETE subcommand when the backup identifier or name is specified with the -i option, and if the current total number of backups for the specified database server is greater than the maximum number of redundancy backups currently specified with the retention_policy parameter. If the total number of backups is less than or equal to the specified, maximum number of redundancy backups, then no additional backups can be deleted using DELETE with the -i backup option.
Under a recovery window retention policy currently set with the retention_policy parameter in the BART configuration file, any backup regardless of its marked status, can be deleted with the DELETE subcommand when the backup identifier or name is specified with the -i option, and if the backup date/time is not within the recovery window currently specified with the retention_policy parameter. If the backup date/time is within the recovery window, then it cannot be deleted using DELETE with the -i backup option.
Invoking the DELETE subcommand with the -i all option results in the deletion of all backups regardless of the retention policy and regardless of whether the status is marked as active, obsolete, or keep.
Note 1: Deletion occurs only if the total number of backups for the specified database server is greater than the specified, maximum number of redundancy backups currently set with the redundancy_policy parameter in the BART configuration file.
Note 2: Deletion occurs only if the backup is not within the recovery window currently set with the redundancy_policy parameter in the BART configuration file.
Use the MANAGE subcommand with the -c keep option to retain such backups indefinitely as shown by the following example:
The next time the MANAGE subcommand is invoked with either no options or with only the -s option, the active backup may be marked as obsolete according to the current retention_policy setting.
Based upon the current number of backups for the database server for a redundancy retention policy or the current date/time for a recovery window retention policy, when the MANAGE subcommand is invoked it evaluates active backups for the database server specified by the -s option or for all database servers if -s all is specified or the -s option is omitted.
Note: The status of backups currently marked as obsolete or keep are not changed. To re-evaluate such backups and then classify them, their status must first be reset back to active with the MANAGE -c nokeep option. See Section 5.2.2 for information.
Invoke the MANAGE subcommand with the -n option to perform a dry run to observe which active backups would be changed to obsolete according to the retention policy:
The dry run shows that backups 1428502049836 and 1428422324880 would be marked as obsolete.
Invoke the MANAGE subcommand omitting the -n option to change and mark the status of the backups as obsolete:
The obsolete backups can be observed in a number of ways. Use the MANAGE subcommand with the -l option to list the obsolete backups:
The STATUS field of the SHOW-BACKUPS subcommand displays the current status:
The details of an individual backup can be displayed using the SHOW-BACKUPS subcommand with the -t option. Note the status in the BACKUP STATUS field.
Use the MANAGE subcommand with the -d option to physically delete the obsolete backups including the unneeded WAL files.
The SHOW-BACKUPS subcommand now displays the remaining backups marked as active or keep:
Invoke the MANAGE subcommand with the -n option to perform a dry run to observe which active backups would be changed to obsolete according to the retention policy.
The dry run shows that backups 1428684544008 and 1428590536488 would be marked as obsolete.
Invoke the MANAGE subcommand omitting the -n option to change and mark the status of the backups as obsolete:
The obsolete backups can be observed in a number of ways. Use the MANAGE subcommand with the -l option to list the obsolete backups:
The STATUS field of the SHOW-BACKUPS subcommand displays the current status:
The details of an individual backup can be displayed using the SHOW-BACKUPS subcommand with the -t option. Note the status in the BACKUP STATUS field.
Use the MANAGE subcommand with the -d option to physically delete the obsolete backups including the unneeded WAL files.
The SHOW-BACKUPS subcommand now displays the remaining backups marked as active or keep:

5 Operation : 5.2 Managing Backups Using a Retention Policy

Table of Contents Previous Next