Mount etcd - New GCP Instance

This procedure is required for all clusters, including remote clusters, and is run on data nodes only.

The Kubernetes etcd database must be on a separate disk to give it access to all the resources required for optimal TOS performance, stability and minimal latency. We recommend performing this procedure after you install the operating system and before you install TOS.

This procedure must be performed by an experienced Linux administrator with knowledge of network and storage configuration.

This procedure is for a stand-alone environment with a single data node. If you have a high availability deployment, see Move etcd - GCP High Availability.

Preliminary Preparations

1. Switch to the root user.

   [<ADMIN> ~]$ sudo su -

   sudo su -

2. Install the rsync RPM.

   [<ADMIN> ~]$ dnf install rsync

   dnf install rsync

3. Find the name of the last disk added to the VM instance.

   [<ADMIN> ~]# lsblk -ndl -o NAME

   lsblk -ndl -o NAME

   The output returns the list of disks on the VM instance. The last letter of the disk name indicates in which it was added, for example: sda, sdb, sdc.

4. Save the name of the last disk in a separate location. You will need it later for verification purposes.

Mount the etcd Database to a Separate Disk

1. Run the tmux command.

   [<ADMIN> ~]$ tmux new-session -s etcd

   tmux new-session -s etcd
2. Add a disk to the VM Instance.

   1. In GCP, go to the VM instance, and on the Details page, click Edit.

   2. Under Additional Disks, click Add new disk.

   3. Configure the following settings:

      • Name:

      • Source Type: Blank

      • Region: Same as VM instance

      • Location: Same as VM instance

      • Disk Type: ssd persistent disk

      • Storage: At least 50 GB

   4. Click Done and then Save.

3. Mount the new disk.

   1. Log into the data node as the root user.

   2. Run the tmux command.

      [<ADMIN> ~]$ tmux new-session -s etcd

      tmux new-session -s etcd

   3. Verify that the new disk is recognized by the operating system.

      [<ADMIN> ~]# lsblk

      lsblk

      [<ADMIN> ~]# ls -l /dev/sd*

      ls -l /dev/sd*

      Compare the output with the name of the disk you saved in the preliminary preparations, and verify that the disk name it returned ends with the next letter in the alphabet. For example, if the name you saved was sdb the output should return sdc. This indicates that the operating system recognizes the new disk.

   4. Create a variable with the block device path of the new disk.

      [<ADMIN> ~]# BLOCK_DEV="/dev/sd<>"

      BLOCK_DEV="/dev/sd<>"

      where <> represents the letter of the new disk.

   5. Generate a UUID for the block device of the new disk.

      [<ADMIN> ~]# BLOCK_UUID="$(uuidgen)"

      BLOCK_UUID="$(uuidgen)"

   6. Create a primary partition on the new disk.

      [<ADMIN> ~]# parted -s -a optimal ${BLOCK_DEV} mklabel msdos -- mkpart primary ext4 1MiB 100%

      parted -s -a optimal ${BLOCK_DEV} mklabel msdos -- mkpart primary ext4 1MiB 100%

   7. Verify that the partition was created.

      [<ADMIN> ~]# parted -s ${BLOCK_DEV} print

      parted -s ${BLOCK_DEV} print

   8. Format the partition as ext4.

      [<ADMIN> ~]# mkfs.ext4 -L ETCD -U ${BLOCK_UUID} ${BLOCK_DEV}1

      mkfs.ext4 -L ETCD -U ${BLOCK_UUID} ${BLOCK_DEV}1

   9. Verify that the partition has been formatted with the UUID and the etcd label (output should return the partition with UUID and an ETCD label).

      [<ADMIN> ~]# blkid | grep "$BLOCK_UUID"

      blkid | grep "$BLOCK_UUID"

   10. Create the mount point of the etcd database.

       [<ADMIN> ~]# mkdir -p /var/lib/rancher/k3s/server/db

       mkdir -p /var/lib/rancher/k3s/server/db

   11. Set the partition to mount upon operating system startup.

       [<ADMIN> ~]# echo "UUID=${BLOCK_UUID} /var/lib/rancher/k3s/server/db ext4 defaults 0 0" >> /etc/fstab

       echo "UUID=${BLOCK_UUID} /var/lib/rancher/k3s/server/db ext4 defaults 0 0" >> /etc/fstab

   12. Load the changes to the filesystem.

       [<ADMIN> ~]# systemctl daemon-reload

       systemctl daemon-reload

   13. Mount the partition that was added to /etc/fstab.

       [<ADMIN> ~]# mount /var/lib/rancher/k3s/server/db

       mount /var/lib/rancher/k3s/server/db

       If the output is not empty, stop the procedure. The etcd disk cannot be mounted. Review what was missed in the previous steps.

   14. Verify the partition has been mounted (the output should return the block device and mount point).

       [<ADMIN> ~]# mount | grep "/var/lib/rancher/k3s/server/db"

       mount | grep "/var/lib/rancher/k3s/server/db"

       If the output is empty, stop the procedure. The etcd disk is not mounted. Review what was missed in the previous steps.

   15. You can now safely exit the tmux session:

       [<ADMIN> ~]# exit

       exit
4. Install TOS Aurora.

   See Clean Install of TOS Aurora on GCP

5. Check the cluster status.

   1. On the primary data nodes, check the TOS status.

      [<ADMIN> ~]$ sudo tos status

      sudo tos status

   2. In the output, check if the System Status is Ok and all the items listed under Components appear as ok. If this is not the case, contact Tufin Support.

   Example output:

   [<ADMIN> ~]$ sudo tos status
    Tufin Orchestration Suite 2.0
   
    System Status: Ok
    System Mode:   Multi Node
   
    Nodes:
      1 Master, 1 Worker. Total 2 nodes. Nodes are healthy.
   
    Components:
      Node:            Ok
      Cassandra:       Ok
      Mongodb:         Ok
      Mongodb_sc:      Ok
      Nats:            Ok
      Neo4j:           Ok
      Postgres:        Ok
      Postgres_sc:     Ok