Update TufinOS 4.x to 4.30

Overview

This procedure is for updating TufinOS 4 to the latest version.

TufinOS updates are additions to the current version of the operating system. Unlike upgrades, where you replace the operating system completely, an update is used for enhancing TufinOS with the latest security and performance features as well as address any issues in order to provide a better working experience.

This procedure does not require reinstalling TOS.

The type of procedure you need to perform will depend on your deployment:

High Availability:
- Without downtime - Update the worker nodes, and then update TufinOS on each data node separately. For more information on HA, see High Availability.
  
  After updating a data node, run tos status and check if the System Status is ok and all the items listed under Components appear as ok. If this is not the case, wait for the database to sync before proceeding to update the next node.
- With downtime - Power down TOS on all nodes in the cluster and then proceed to update TufinOS on all your nodes.
Single data node cluster: First update the worker nodes and then update the data node. During the update itself, there will be some downtime as all TOS processes will need to be stopped and then restarted.
High Availability Central Cluster + Remote Collector clusters:
- Central Cluster - See High Availability bullet above
- Remote Collector Clusters - First update the worker nodes, and then update TufinOS on the data node. Repeat for each remote cluster. For more information on Remote Collector clusters, see Remote Collectors.
High availability is not supported for Remote Collector clusters.
Single data node cluster + Remote Collector clusters: First update the central cluster: worker nodes and then data node. Afterwards, repeat for each Remote Collector cluster.

Some steps are meant to be performed only on the primary data node while others are also for worker nodes, if present on your cluster.

Preliminary Preparations

Verify that your deployment is compatible with TufinOS 4.30
1. Check your current TufinOS version:
  cat /etc/tufinos-release
  Your TufinOS release appears in the output. If the command is not recognized, you have a non-TufinOS operating system and cannot upgrade using this procedure.
2. Review the compatibility and requirements and supported upgrade paths for TufinOS 4.30 and verify that you can perform the procedure.

Check the cluster health.

On the primary data node, check the following status.
```
[<ADMIN> ~]$ sudo systemctl status k3s
```
sudo systemctl status k3s

In the output under the line k3s.service - Aurora Kubernetes, two lines should appear - Loaded... and Active... similar to the example below. If they appear, continue with the next step, otherwise contact Tufin Support for assistance.

Example output:

[<ADMIN> ~]$ sudo systemctl status k3s
 [root@TufinOS ~]# systemctl status k3s
 Redirecting to /bin/systemctl status k3s.service
 ● k3s.service - Aurora Kubernetes
    Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: disabled)
    Active: active (running) since Tue 2021-08-24 17:14:38 IDT; 1 day 18h ago
      Docs: https://k3s.io
   Process: 1241 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Process: 1226 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
  Main PID: 1250 (k3s-server)
     Tasks: 1042
    Memory: 2.3G

On the primary data node, check the TOS status.
```
[<ADMIN> ~]$ tos status
```
tos status
In the output, check if the System Status is Ok and all the items listed under Components appear as Ok. If this is not the case, contact Tufin Support.

Example output for a central cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.1.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Multi Node"

Nodes
  Nodes:
  - ["node1"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  - ["node3"]
    Type: "Worker Node"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 4%

registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["cassandra"]
  Status: "Ok"
- ["kafka"]
  Status: "Ok"
- ["mongodb"]
  Status: "Ok"
- ["mongodb_sc"]
  Status: "Ok"
- ["ongDb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"
- ["postgres_sc"]
  Status: "Ok"

Application
Application Services Status OK
Running services 50/50

Remote Clusters
Number Of Remote Clusters: 2
  - ["RC"]
     Connectivity Status:: "OK:"
  - ["RC2"]
     Connectivity Status:: "OK"

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Example output for a remote cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.0.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Single Node"

Nodes
  Nodes:
  - ["node2"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  
registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["mongodb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"

Application
Application Services Status OK
Running services 16/16

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Remote Collector clusters. If you are currently updating the central cluster, make sure all the Remote Collector clusters are connected.
On the Central cluster primary data node run:
[<ADMIN> ~]$ sudo tos cluster list
sudo tos cluster list
The output shows all connected clusters.

Example output:
$ sudo tos cluster list NAME ID IP TYPE Central 1 master RC1 2 192.168.75.156 data_collector

Downloads

Download the TufinOS 4.30 update package from the Download Center to your local machine.
Log in to the node you are updating as the tufin-admin user.
Transfer the TufinOS update package to /opt/misc.
Extract the TOS run file the archive.
```
[<ADMIN> ~]$ sudo tar xzvf <FILENAME>.tgz
```
sudo tar xzvf <FILENAME>.tgz
The run file name includes the release, version, and build number.

TufinOS update file example: TufinOS-4.20-639387-x86_64-8.8-Final-Update.run.tgz
Verify the integrity of the TufinOS installation package.
```
[<ADMIN> ~]# sha256sum -c TufinOS-4.20-639387-x86_64-8.8-Final-Update.run.sha256
```
sha256sum -c TufinOS-4.20-639387-x86_64-8.8-Final-Update.run.sha256
The output should return OK

Update TufinOS

Update TufinOS in a High Availability Cluster

Update without downtime
- Update worker nodes
  1. Transfer the TufinOS update file <update_file> to the node you are upgrading, to directory /opt/misc.
  2. Switch to the root user
    
    [<ADMIN> ~]$ sudo su -
    
    sudo su -
  3. Go to /opt/misc.
    
    [<ADMIN> ~]# cd /opt/misc
    
    cd /opt/misc
  4. Run the screen command.
    
    [<ADMIN> ~]# screen -S update
    
    sudo screen -S update
  5. Execute the TufinOS update file:
    
    [<ADMIN> ~]# sudo sh <update_file>.run
    
    sudo sh <update_file>.run
  6. When prompted to continue the update, enter yes.
    
    Do not interrupt the update process. Wait until the successful completion message appears.
  7. After the update is complete, reboot the node:
    
    [<ADMIN> ~]# sudo reboot
    
    sudo reboot
  8. Log in to the primary data node.
  9. Check the TOS Status
    
    On the primary data node, check the TOS status.
    
    [<ADMIN> ~]$ sudo tos status
    
    sudo tos status
    
    In the output, check if the System Status is Ok and all the items listed under Components appear as ok. If this is not the case, contact Tufin Support.
    
    Example output:
    [<ADMIN> ~]$ tos status [Mar 28 13:42:09] INFO Checking cluster health status TOS Aurora Tos Version: 24.2 (PRC1.1.0) System Status: "Ok" Cluster Status: Status: "Ok" Mode: "High Availability" Nodes Nodes: - ["node1"] Type: "Primary" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 32% - ["node3"] Type: "Ha Data Node" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 11% - ["node2"] Type: "Ha Data Node" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 11% registry Expiration ETA: 819 days Status: "Ok" Infra Databases: - ["cassandra"] Status: "Ok" - ["kafka"] Status: "Ok" - ["mongodb"] Status: "Ok" - ["mongodb_sc"] Status: "Ok" - ["ongDb"] Status: "Ok" - ["postgres"] Status: "Ok" - [postgres_sc"] Status: "Ok" Application Application Services Status OK Running services 54/54 Backup Storage: Location: "Local s3:http://minio.default.svc:9000/velerok8s/restic/default " Status: "Ok" Latest Backup: 2024-03-23 05:00:34 +0000 UTC
- Update data nodes
  Repeat this procedure for all three data nodes. First update the non-primary data nodes and then update the primary data node.
  
  After updating a data node, run tos status and check if the System Status is ok and all the items listed under Components appear as ok. If this is not the case, wait for the database to sync before proceeding to update the next node.
  1. Backup your TOS data. This needs to be performed for all primary data nodes - the central cluster and remote collector clusters.
    
    If you are going to perform this procedure over multiple maintenance periods, create a new backup each time.
    
    Create the backup using tos backup create:
    
    [<ADMIN> ~]$ sudo tos backup create
    
    sudo tos backup create
    
    Example output:
    [%=Local.admin-prompt% sudo tos backup create [Aug 23 16:18:42] INFO Running backup Backup status can be monitored with "tos backup status"
    
    You can check the backup creation status using tos backup status, which shows the status of backups in progress. Wait until completion before continuing.
    
    [<ADMIN> ~]$ sudo tos backup status
    
    sudo tos backup status
    
    Example output:
    [<ADMIN> ~]$ sudo tos backup status Found active backup "23-august-2021-16-18"
    
    Run the following command to display the list of backups saved on the node:
    
    [<ADMIN> ~]$ sudo tos backup list
    
    sudo tos backup list
    
    Example output:
    [<ADMIN> ~]$ sudo tos backup list ["23-august-2021-16-18"] Started: "2021-08-23 13:18:43 +0000 UTC" Completed: "N/A" Modules: "ST, SC" HA mode: "false" TOS release: "21.2 (PGA.0.0) Final" TOS build: "21.2.2100-210722164631509" Expiration Date: "2021-09-22 13:18:43 +0000 UTC" Status: "Completed"
    
    Check that your backup file appears in the list, and that the status is "Completed".
    
    If your backup files are saved locally:
    
    Run sudo tos backup export to save your backup file from a TOS backup directory as a single .gzip file. If there are other backups present, they will be included as well.
    
    Transfer the exported .gzip file to a safe, remote location.
    
    Make sure you have the location of your backups safely documented and accessible, including credentials needed to access them, for recovery when needed.
    
    After the backup is exported, we recommend verifying that the file contents can be viewed by running the following command:
    
    [Target location]$ tar tzvf <filename>
    
    tar tzvf <file name>
  2. Transfer the TufinOS update file <update_file> to the node you are upgrading, to directory /opt/misc.
  3. Get the name of the node you want to update by running the following command on any of the data nodes.
    
    kubectl get node
    
    A list of nodes which currently exist in the cluster is displayed.
  4. In the output, note the name of the node you want to update.
  5. Run the following command:
  6. Switch to the root user
    
    [<ADMIN> ~]$ sudo su -
    
    sudo su -
  7. Go to /opt/misc.
    
    [<ADMIN> ~]# cd /opt/misc
    
    cd /opt/misc
  8. Run the screen command.
    
    [<ADMIN> ~]# screen -S update
    
    sudo screen -S update
  9. Execute the TufinOS update file:
    
    [<ADMIN> ~]# sudo sh <update_file>.run
    
    sudo sh <update_file>.run
  10. When prompted to continue the update, enter yes.
    
    Do not interrupt the update process. Wait until the successful completion message appears.
  11. After the update is complete, run the following command:
  12. After the update is complete, reboot the node:
    
    [<ADMIN> ~]# sudo reboot
    
    sudo reboot
  13. Log in to the primary data node.
  14. Check the TOS Status
    
    On the primary data node, check the TOS status.
    
    [<ADMIN> ~]$ sudo tos status
    
    sudo tos status
    
    In the output, check if the System Status is Ok and all the items listed under Components appear as ok. If this is not the case, contact Tufin Support.
    
    Example output:
    [<ADMIN> ~]$ tos status [Mar 28 13:42:09] INFO Checking cluster health status TOS Aurora Tos Version: 24.2 (PRC1.1.0) System Status: "Ok" Cluster Status: Status: "Ok" Mode: "High Availability" Nodes Nodes: - ["node1"] Type: "Primary" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 32% - ["node3"] Type: "Ha Data Node" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 11% - ["node2"] Type: "Ha Data Node" Status: "Ok" Disk usage: - ["/opt"] Status: "Ok" Usage: 11% registry Expiration ETA: 819 days Status: "Ok" Infra Databases: - ["cassandra"] Status: "Ok" - ["kafka"] Status: "Ok" - ["mongodb"] Status: "Ok" - ["mongodb_sc"] Status: "Ok" - ["ongDb"] Status: "Ok" - ["postgres"] Status: "Ok" - [postgres_sc"] Status: "Ok" Application Application Services Status OK Running services 54/54 Backup Storage: Location: "Local s3:http://minio.default.svc:9000/velerok8s/restic/default " Status: "Ok" Latest Backup: 2024-03-23 05:00:34 +0000 UTC
Update with downtime
1. Backup your TOS data. This needs to be performed for all primary data nodes - the central cluster and remote collector clusters.
  
  If you are going to perform this procedure over multiple maintenance periods, create a new backup each time.
  
  Create the backup using tos backup create:
  
  [<ADMIN> ~]$ sudo tos backup create
  
  sudo tos backup create
  
  Example output:
  [%=Local.admin-prompt% sudo tos backup create [Aug 23 16:18:42] INFO Running backup Backup status can be monitored with "tos backup status"
  
  You can check the backup creation status using tos backup status, which shows the status of backups in progress. Wait until completion before continuing.
  
  [<ADMIN> ~]$ sudo tos backup status
  
  sudo tos backup status
  
  Example output:
  [<ADMIN> ~]$ sudo tos backup status Found active backup "23-august-2021-16-18"
  
  Run the following command to display the list of backups saved on the node:
  
  [<ADMIN> ~]$ sudo tos backup list
  
  sudo tos backup list
  
  Example output:
  [<ADMIN> ~]$ sudo tos backup list ["23-august-2021-16-18"] Started: "2021-08-23 13:18:43 +0000 UTC" Completed: "N/A" Modules: "ST, SC" HA mode: "false" TOS release: "21.2 (PGA.0.0) Final" TOS build: "21.2.2100-210722164631509" Expiration Date: "2021-09-22 13:18:43 +0000 UTC" Status: "Completed"
  
  Check that your backup file appears in the list, and that the status is "Completed".
  
  If your backup files are saved locally:
  
  Run sudo tos backup export to save your backup file from a TOS backup directory as a single .gzip file. If there are other backups present, they will be included as well.
  
  Transfer the exported .gzip file to a safe, remote location.
  
  Make sure you have the location of your backups safely documented and accessible, including credentials needed to access them, for recovery when needed.
  
  After the backup is exported, we recommend verifying that the file contents can be viewed by running the following command:
  
  [Target location]$ tar tzvf <filename>
  
  tar tzvf <file name>
2. Shut down TOS.
  
  Run the command:
  
  [<ADMIN> ~]$ sudo tos stop
  
  sudo tos stop
  
  This process may take time.
  
  Check that all processes have been stopped successfully. Run the command:
  
  [<ADMIN> ~]$ watch -n 1 kubectl get pods
  
  watch -n 1 kubectl get pods
  
  A list of all pods is displayed.
  
  Example
  
  Wait until all the pods, with the exception of the service controller, ps-proxy, and reportpack pods, have disappeared from the list or reached a status of Completed. The service controller, ps-proxy, and reportpack pods which can continue running.
  
  All TOS processes are now stopped on all the data nodes in the cluster.
3. Repeat steps 4-8 for each node in the cluster.
4. Transfer the TufinOS update file <update_file> to the node you are upgrading, to directory /opt/misc.
5. Switch to the root user
  [<ADMIN> ~]$ sudo su -
  
  sudo su -
6. Go to /opt/misc.
  [<ADMIN> ~]# cd /opt/misc
  
  cd /opt/misc
7. Run the screen command.
  [<ADMIN> ~]# screen -S update
  
  sudo screen -S update
8. Execute the TufinOS update file:
  [<ADMIN> ~]# sudo sh <update_file>.run
  
  sudo sh <update_file>.run
9. When prompted to continue the update, enter yes.
  
  Do not interrupt the update process. Wait until the successful completion message appears.
10. After the update is complete, reboot the node:
  [<ADMIN> ~]# sudo reboot
  
  sudo reboot
11. After updating all data nodes, log in to the primary data node.
12. Restart TOS on the primary data node:
  [<ADMIN> ~]$ sudo tos start
  
  sudo tos start
13. Check the TOS Status
  1. On the primary data node, check the TOS status.
    
    [<ADMIN> ~]$ sudo tos status
    
    sudo tos status
  2. In the output, check if the System Status is Ok and all the items listed under Components appear as ok. If this is not the case, contact Tufin Support.

Update TufinOS in Single Data Node Cluster

Update worker nodes

Transfer the TufinOS update file <update_file> to the node you are upgrading, to directory /opt/misc.
Switch to the root user
```
[<ADMIN> ~]$ sudo su -
```
sudo su -
Go to /opt/misc.
```
[<ADMIN> ~]# cd /opt/misc
```
cd /opt/misc
Run the screen command.
```
[<ADMIN> ~]# screen -S update
```
sudo screen -S update
Execute the TufinOS update file:
```
[<ADMIN> ~]# sudo sh <update_file>.run
```
sudo sh <update_file>.run
When prompted to continue the update, enter yes.

Do not interrupt the update process. Wait until the successful completion message appears.
After the update is complete, reboot the node:
```
[<ADMIN> ~]# sudo reboot
```
sudo reboot
Log in to the primary data node.

Check the TOS Status

On the primary data node, check the TOS status.
```
[<ADMIN> ~]$ tos status
```
tos status
In the output, check if the System Status is Ok and all the items listed under Components appear as Ok. If this is not the case, contact Tufin Support.

Example output for a central cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.1.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Multi Node"

Nodes
  Nodes:
  - ["node1"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  - ["node3"]
    Type: "Worker Node"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 4%

registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["cassandra"]
  Status: "Ok"
- ["kafka"]
  Status: "Ok"
- ["mongodb"]
  Status: "Ok"
- ["mongodb_sc"]
  Status: "Ok"
- ["ongDb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"
- ["postgres_sc"]
  Status: "Ok"

Application
Application Services Status OK
Running services 50/50

Remote Clusters
Number Of Remote Clusters: 2
  - ["RC"]
     Connectivity Status:: "OK:"
  - ["RC2"]
     Connectivity Status:: "OK"

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Example output for a remote cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.0.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Single Node"

Nodes
  Nodes:
  - ["node2"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  
registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["mongodb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"

Application
Application Services Status OK
Running services 16/16

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Update primary data node

Backup your TOS data. This needs to be performed for all primary data nodes - the central cluster and remote collector clusters.
If you are going to perform this procedure over multiple maintenance periods, create a new backup each time.
1. Create the backup using tos backup create:
2. You can check the backup creation status using tos backup status, which shows the status of backups in progress. Wait until completion before continuing.
3. Run the following command to display the list of backups saved on the node:
  [<ADMIN> ~]$ sudo tos backup list
  
  sudo tos backup list
4. Check that your backup file appears in the list, and that the status is "Completed".
5. If your backup files are saved locally:
  1. Run sudo tos backup export to save your backup file from a TOS backup directory as a single .gzip file. If there are other backups present, they will be included as well.
  2. Transfer the exported .gzip file to a safe, remote location.
    
    Make sure you have the location of your backups safely documented and accessible, including credentials needed to access them, for recovery when needed.
  After the backup is exported, we recommend verifying that the file contents can be viewed by running the following command:
  [Target location]$ tar tzvf <filename>
  
  tar tzvf <file name>
Shut down TOS.
1. Run the command:
  [<ADMIN> ~]$ sudo tos stop
  
  sudo tos stop
  This process may take time.
2. Check that all processes have been stopped successfully. Run the command:
3. Wait until all the pods, with the exception of the service controller, ps-proxy, and reportpack pods, have disappeared from the list or reached a status of Completed. The service controller, ps-proxy, and reportpack pods which can continue running.
  
  All TOS processes are now stopped on all the data nodes in the cluster.
Transfer the TufinOS update file <update_file> to the node you are upgrading, to directory /opt/misc.
Switch to the root user
```
[<ADMIN> ~]$ sudo su -
```
sudo su -
Go to /opt/misc.
```
[<ADMIN> ~]# cd /opt/misc
```
cd /opt/misc
Run the screen command.
```
[<ADMIN> ~]# screen -S update
```
sudo screen -S update
Execute the TufinOS update file:
```
[<ADMIN> ~]# sudo sh <update_file>.run
```
sudo sh <update_file>.run
When prompted to continue the update, enter yes.

Do not interrupt the update process. Wait until the successful completion message appears.
After the update is complete, reboot the node:
```
[<ADMIN> ~]# sudo reboot
```
sudo reboot
Log in to the primary data node.
Restart TOS on the primary data node:
```
[<ADMIN> ~]$ sudo tos start
```
sudo tos start

All TOS processes will be restarted on all the data nodes in the cluster, and you will be able to resume using TOS.

Check the TOS Status

On the primary data node, check the TOS status.
```
[<ADMIN> ~]$ tos status
```
tos status
In the output, check if the System Status is Ok and all the items listed under Components appear as Ok. If this is not the case, contact Tufin Support.

Example output for a central cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.1.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Multi Node"

Nodes
  Nodes:
  - ["node1"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  - ["node3"]
    Type: "Worker Node"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 4%

registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["cassandra"]
  Status: "Ok"
- ["kafka"]
  Status: "Ok"
- ["mongodb"]
  Status: "Ok"
- ["mongodb_sc"]
  Status: "Ok"
- ["ongDb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"
- ["postgres_sc"]
  Status: "Ok"

Application
Application Services Status OK
Running services 50/50

Remote Clusters
Number Of Remote Clusters: 2
  - ["RC"]
     Connectivity Status:: "OK:"
  - ["RC2"]
     Connectivity Status:: "OK"

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Example output for a remote cluster data node:

[<ADMIN> ~]$ tos status         
[Mar 28 13:42:09]  INFO Checking cluster health status           
TOS Aurora
Tos Version: 24.2 (PRC1.0.0)

System Status: "Ok"
            
Cluster Status:
   Status: "Ok"
   Mode: "Single Node"

Nodes
  Nodes:
  - ["node2"]
    Type: "Primary"
    Status: "Ok"
    Disk usage:
    - ["/opt"]
      Status: "Ok"
      Usage: 19%
  
registry
  Expiration ETA: 819 days
  Status: "Ok"

Infra
Databases:
- ["mongodb"]
  Status: "Ok"
- ["postgres"]
  Status: "Ok"

Application
Application Services Status OK
Running services 16/16

  Backup Storage:
  Location: "Local
s3:http://minio.default.svc:9000/velerok8s/restic/default "
  Status: "Ok"
  Latest Backup: 2024-03-23 05:00:34 +0000 UTC

Update TufinOS in a High Availability Cluster with Remote Collector Clusters

Follow the procedures above to update the nodes in the Central cluster and the Remote Collector clusters.

After updating all nodes:

Check that the Remote Collector is still connected.

Run the following command:

[<ADMIN> ~]$ sudo tos cluster list

sudo tos cluster list

The output shows all connected clusters.

Example output:

$ sudo tos cluster list
NAME    ID    IP                TYPE

Central    1                    master
RC1        2    192.168.75.156    data_collector

If the Remote Collector cluster is not connected, run the following commands to connect it to the Central cluster, where:

[<ADMIN> ~]$ sudo tos cluster connect --central-cluster-vip=CENTRAL-CLUSTER-VIP --remote-cluster-vip=REMOTE-CLUSTER-VIP --remote-cluster-name=REMOTE-CLUSTER-NAME --initial-secret=OTP

sudo tos cluster connect --central-cluster-vip=CENTRAL-CLUSTER-VIP --remote-cluster-vip=REMOTE-CLUSTER-VIP --remote-cluster-name=REMOTE-CLUSTER-NAME --initial-secret=OTP

where

Parameter	Description	Required/Optional
`--central-cluster-vip`	External IP address (VIP) of your central server cluster.	Required
`--remote-cluster-vip`	External IP address (VIP) of the server you want to connect (i.e. the current server).	Required
`--remote-cluster-name`	Any alphanumeric string you choose; quotes are not used so you cannot embed spaces.	Required
`--initial-secret`	One-time password returned from running tos cluster generate-otp on the central server.	Required

Update TufinOS in Single Data Node Cluster with Remote Collector Clusters

Follow the procedures above to update the nodes in the Central cluster and the Remote Collector clusters.

After updating all nodes:

Check that the Remote Collector is still connected.

Run the following command:

[<ADMIN> ~]$ sudo tos cluster list

sudo tos cluster list

The output shows all connected clusters.

Example output:

$ sudo tos cluster list
NAME    ID    IP                TYPE

Central    1                    master
RC1        2    192.168.75.156    data_collector

If the Remote Collector cluster is not connected, run the following commands to connect it to the Central cluster, where:

[<ADMIN> ~]$ sudo tos cluster connect --central-cluster-vip=CENTRAL-CLUSTER-VIP --remote-cluster-vip=REMOTE-CLUSTER-VIP --remote-cluster-name=REMOTE-CLUSTER-NAME --initial-secret=OTP

sudo tos cluster connect --central-cluster-vip=CENTRAL-CLUSTER-VIP --remote-cluster-vip=REMOTE-CLUSTER-VIP --remote-cluster-name=REMOTE-CLUSTER-NAME --initial-secret=OTP

where

Parameter	Description	Required/Optional
`--central-cluster-vip`	External IP address (VIP) of your central server cluster.	Required
`--remote-cluster-vip`	External IP address (VIP) of the server you want to connect (i.e. the current server).	Required
`--remote-cluster-name`	Any alphanumeric string you choose; quotes are not used so you cannot embed spaces.	Required
`--initial-secret`	One-time password returned from running tos cluster generate-otp on the central server.	Required