On This Page
Backup and Restore
Overview
TOS Aurora uses a Point in Time Recovery (PITR) backup methodology, which substantially reduces the time and disk space required to create a complete daily backup and allows you to restore your data to a specified point in time in a single step. You can define a backup policy, in which you specify the frequency of automated backups that will run without the need for administrator maintenance. This is the recommended method of handling TOS Aurora backups. In addition, on-demand backups can be made, unconnected to your policy.
All backup and restore functions apply equally to central clusters and remote collectors. The backup will always apply to the deployment type as specified in the modules parameter when the install command was run. A backup taken from one kind of cluster must be restored to the same type. For example, a central cluster must only be restored from a central cluster backup and a remote collector must only be restored from a remote collector backup. Otherwise; data may be lost or corrupted.
Zero Touch Automated Backups
Once you create a policy, TOS Aurora will take care of everything else. This mechanism does not have the concepts of full backup (e.g. weekly, monthly) and sequential backup (e.g. daily) because every backup is an image, made up of pointers to saved 'blocks' of your original and modified data. The image file is effectively a full backup and running the restore commands on it will rebuild your database to the appropriate point in time from the saved blocks .
Reduced Disk Space
Backups, which use this concept, duplicate all your original data in a new location one time only. After the initial backup, only changed blocks are added to a steadily growing pool, and pointers in the backup file refer to these blocks. Subsequent backups have no need to refer to blocks that have since been replaced, while prior backups refer only to the blocks needed to reconstruct the data in a restore operation, and not those that changed subsequent to the backup execution.
Cleanup
Once initiated, the backup policy runs a cleanup function in the background, in which blocks no longer referred to by any backups are deleted. This situation will occur when backup files reach their expiry time and a block referred to only by this backup has since been replaced with another.
VM Snapshots
We do not support using VM snapshots as a backup mechanism for TOS data and using this method may lead to undesired results. We strongly recommend using the TOS CLI backup/restore commands only.
What Gets Backed up
Included: All TOS data, TOS configurations, data for Tufin extensions (formerly Tufin Marketplace apps), certificates that you have imported.
Not included: TOS software, backup policy, Tufin auto-generated certificates and certificates for communicating between central and remote collector clusters.
Tufin auto-generated certificates will be automatically re-created on the first connection following a restore and central and remote collector clusters will need to be reconnected manually as explained in Remote Collectors.
Backup File Name and Location
By default, the name and location of backup files are fixed, with all backups being placed in the TOS backup directory /opt/tufin/data/volumes/minio/velerok8s/backups
on the primary data node.
-
On-demand backups are named automatically in the format
DD-Month-HH-MM
. For example, "5-november-2019-19-32" for November 5th 2019 at 19:32 -
Scheduled backups initiated from your backup policy are named in the format
tufin-scheduler-yyyymmddhhmmss
. For example, "tufin-scheduler-20191110000045" for November 10th 2019 at 00:00:45 (45 seconds past midnight).
All backup files are created with a time-to-live parameter. When the time to live expires, the backup will be destroyed automatically and it's disk space released. If no value is specified, the default value of 720 hours will be used. Backup files cannot be set to never expire, but the maximum value is 2,147,483,647 (245 thousand years), which should be enough for most users!
There are two approaches to store backups to a safe environment - Local (default) and External. Local backups are placed in the TOS backup directory on the primary data node. External backups are stored outside of the cluster. To change the default location to external
, use sudo tos backup storage set and set the location to external
. This command changes the location of the backup file according to the selected provider. To change the backup storage location back to the primary data node, you can use the same command - sudo tos backup storage set - and set the location to local
.
Local Backup Storage
Local backups reside only on the primary data node, regardless of the cluster configuration. Disaster recovery planning dictates that you should separate these backups from the cluster. We recommend that you export your backups, which saves all accumulated backups to a single file. Then, you can copy this file to a safe, remote location.
External Backup Storage
External backups reside in a location outside of the cluster, according to the selected provider. TOS Aurora supports these external backup providers:
-
AWS
-
Azure
-
GCP
-
Minio - You can also deploy Minio on prem and use it externally. See Minio official documentation.
-
Network attached storage (NAS) using the network file system (NFS) protocol
Read/write permissions must be granted to the external backup location.
When you change the backup storage to external
, all new backups will be placed in the specified location. If you have previously run the backup command with the backup location set to local
, the existing backups will remain on the primary data node.
You must confirm that there is sufficient network throughput to the external backup storage bucket; otherwise, backup and restore will take a long time to complete.
Scheduled Backup Procedure
We recommend the following best practice for backing up regularly:
-
Decide on your backup location:
local
to create the backup on your cluster orexternal
to backup directly to a remote location. Use command - sudo tos backup storage set.If the backup location is local: When the /opt partition becomes 90% full, TOS will stop making automatic backups and core services will be stopped. We recommend configuring TOS monitoring to send notifications if too much storage is being consumed.
-
Create a backup policy using sudo tos backup policy set to get TOS to automatically backup your data regularly to the backup location.
-
If the backup location is
local
, we recommend that you export your backups regularly to a remote location.-
Run sudo tos backup export to save all your backups from the TOS backup directory as a single
.gzip
file. -
Transfer the exported
.gzip
file to a remote location. -
Make sure you have the location of your backups safely documented and accessible, including credentials needed to access them, for recovery when needed.
-
-
To restore your data, see Restore Procedure.
One-Time Backup Procedure
We recommend the following procedure for one-time, one-off backups.
-
If you have not set a backup location, use command - sudo tos backup storage set:
local
to create the backup on your cluster orexternal
to backup directly to a remote location. -
Create the backup using sudo tos backup create.
-
Monitor the backup status using sudo tos backup status, which shows the status of backups in progress and sudo tos backup list, which lists all completed backups. Wait until completion before continuing.
-
When sudo tos backup storage set location is
local
:-
Run sudo tos backup export regularly to save all your backups from the TOS backup directory as a single
.gzip
file. If there are other backups present, they will be included as well. -
Transfer the exported
.gzip
file to a remote location after creation. -
Make sure you have the location of your backups safely documented and accessible, including credentials needed to access them, for recovery when needed.
-
-
To restore your data, see Restore Procedure.
Restore Procedure
This procedure is for restoring data from either one-time or scheduled TOS Aurora backups.
Backups can only be restored to a TOS deployment with the same release and version from which the backup was made. For example, a backup of R23-1 PHF2.1.0 can only be restored to an R23-1 PHF2.1.0 deployment.
-
Restore from a backup saved to a remote location using tos backup export
-
Copy the export file (
gzip
format) from the remote location to your primary data node. -
Run sudo tos backup import, specifying the full path of the export file, to extract all the backups from the export file to the TOS backup directory.
-
Run sudo tos backup list to list all the backups and identify the one that you want to restore.
-
Run sudo tos restore specifying the desired backup. On completion, your data will have been restored.
-
-
Restore from a backup made directly to an external backup location
-
Verify that you are connected to the external backup location where the backup was made.
-
Run sudo tos backup list to list all the backups and identify the one that you want to restore.
-
Run sudo tos restore specifying the desired backup. On completion, your data will have been restored.
-
If the cluster was backed up with a different primary VIP, after the restore, change the primary VIP using tos cluster primary-vip set.
Backup Restore Commands
If your system runs on TufinOS, all TOS commands must be run using sudo
, because user root is not available on this operating system. Before running any of these commands, switch the current directory to /opt
.
-
sudo tos backup create - create an on-demand backup
-
sudo tos backup delete - delete a single backup
-
sudo tos backup expiry - sets an expiration date for the backup
-
sudo tos backup export - export all backup files from the TOS backup directory to a remote location
-
sudo tos backup import - Import all exported backup files from a remote location to the TOS backup directory
-
sudo tos backup list - list all backups
-
sudo tos backup policy delete - delete a backup policy
-
sudo tos backup policy get - display a backup policy
-
sudo tos backup policy set - create a backup policy
-
sudo tos restore - restore data from a single backup file
-
sudo tos backup status - show status of currently running backups
-
sudo tos backup storage set - define a backup location
All commands can be run with any of the following optional parameters:
- --help - provides explanation of the command and its parameters without executing
- --debug - executes the command and displays system information