Set Up AWS

Proceed only if...

  1. You have completed all the checks on your current TOS Classic system and the results indicate you are ready to set up your target machine.
  2. Your target platform meets all general and platform-specific prerequisites.
  3. You have available all IP, DNS and other information needed to set up the server and TOS Aurora. We recommended putting this information in your worksheet and sharing with relevant administrators.

Otherwise, go back.

In this step you will:

  • Prepare an AWS instance ready to install TOS Aurora

Overview

This procedure is for the setting up of an AWS instance ready to create a TOS Aurora cluster on it or to add it as a node to an existing TOS Aurora cluster.

Prerequisites

  • You must know the resources you will need - CPU cores, RAM, disk space and the load-model parameter to use in the install command, all of which can be obtained from your account team, based on the procedure Calculate resources - clean install.

  • All resources need to be dedicated to the TOS Aurora machine. Do not use shared CPU or memory and if the datastore is shared, the disk performance must meet the requirements at all times.

  • Do not install any software on your server before or after the deployment of TOS Aurora that is not specified in the current procedure.

  • There are some load limitations for deploying on cloud. Check with your account team that deployment on this platform is supported for your load model before going ahead.

  • A 24-bit CIDR subnet on your network must be dedicated to TOS Aurora for the Kubernetes service network. It must not overlap with:

    • CIDR 10.244.0.0/16, which is already used for Kubernetes internal communication or

    • The physical addresses of your TOS Aurora servers (see below) or

    • Your primary VIP or external loadbalancer IP (see below) or

    • Any other subnets

    If a proxy is configured on your system make sure this network is excluded.

  • Select a storage type of SSD. Take into consideration that TOS requires 7,500 IOPS and the throughput expected will average 250MB/s with bursts of up to 700MB/s.

  • You will need to allow access to required Ports and Services.

  • DNS hostnames must be enabled on your VPC - see Modify the DNS attributes for your VPC (Amazon official documentation)

  • Once TOS Aurora has been installed, changing the host name or IP address will require reinstalling - see Changing IP Address/Host Names. If you want to change the host name of the node, do so before running the tos install command.

  • Partitions:

    You will need to configure three partitions: /opt, /tmp and /var, as well as a separate disk for etcd. The /opt partition will contain your data, which will increase over time. Most of your available disk space should be allocated to this partition and the minimum is determined by the load model parameter (small, medium, large) provided by your account team. Minimum sizes for all partitions:

    Minimum Partition Sizes

    /opt/

    (Small)*

    /opt/

    (Medium)*

    /opt/

    (Large)*

    /tmp/

     

    /var/

     

    etcd
    Central cluster / remote cluster primary data node / HA data nodes 80 GB 170 GB 370 GB 25 GB 200GB 128 GB
    Worker node (central and remote clusters) 70 GB 70 GB 70 GB 25 GB 60 GB N/A

    *Small, medium and large refer to the load model parameter provided by your account team.

    We recommend allocating /opt partition all remaining disk space after you have partitioned the other directories.

Procedure Flow

Complete the steps below in sequence.

Launch the Instance

For additional help, refer to the official AWS documentation - Create your EC2 resources and launch your EC2 instance.

  1. In your AWS console, navigate to EC2 > Instances > Launch Instances.

  2. In the Name and tags pane, enter the name of the instance.

  3. In the Application and OS Images pane, choose an Amazon Machine image (AMI) from the AWS Marketplace. The AMI needs to be for:

    • Red Hat Enterprise Linux 8.6, 8.8, or 8.9 (for R22-2 PHF1.0.0 and later)

    • Rocky Linux 8.6, 8.8, or 8.9 (for R22-2 PHF1.0.0 and later)

    If you select Red Hat, it must be 'Red Hat Enterprise Linux Server Standard'. Other Linux distributions and versions are not supported.

  4. In the Instance type pane, select an instance type that meets your CPU and RAM resource requirements (see Prerequisites section).

  5. In the Key pair (login) pane, select or create a key pair to securely connect to your instance.

  6. In the Network Settings pane, click Edit, and enter/select the following details:

    • Network: The VPC you are using with this instance

    • Subnet: The subnet you are using with this instance

    • Auto-assign public IP: Select Disable.

    • Firewall (security groups): Create a new security group, or select an existing security group that you want to use to control the traffic to your instance.

  7. In the Configure Storage pane:

    1. Click Add new volume.

    2. For each volume, enter/select the following:

      • 300

      • General purpose SSD (gp3)

    3. Click the Advanced link, and set the IOPS, Throughput, and Encryption for each volume. Take into consideration that TOS requires 7,500 IOPS and the throughput expected will average 250MB/s with bursts of up to 700MB/s.

      The encryption should match your company's security policy.

  8. Click Launch Instance.

  9. (Optional) We recommend changing the permissions of the .pem file downloaded to your PC to prevent unauthorized users from running it. If your PC is running on a Linux-like operating system, run the command:

    [<ADMIN> ~]# chmod 400 <pem_key_name>
    chmod 400 <pem_key_name>

  10. When required, log in to the instance as follows:

    [<ADMIN> ~]# ssh -i <pem_key_name> <awsuser>@<IP>
    ssh -i <pem_key_name> <awsuser>@<IP>

    where

    • <pem_key_name> is the name of the .pem file downloaded previously from the AWS console

    • <awsuser> is the name of your AWS user

    • <IP> is its private or public IP

Create Target Groups

Target Ports

After launching the instance, you need to create a target group for the ports you are going to need. These ports are listed in the Target column in the table below. The target groups are rules that redirect traffic to the load balancer.

Protocol

Source

Target

Purpose
TCP 443 31443

Mandatory
TCP 61617 31617

Remote collector connectivity
TCP 9099 31099

OPM devices
TCP 8443 31843

Remote collector connectivity

TCP

6514

31514

TCP syslogs
UDP 514 30514

UDP syslogs
UDP 161 30161

SNMP monitoring
UDP 10161 31161

SNMP monitoring

Create a Target Group

Repeat this procedure for each port you need.

  1. In your AWS console, navigate to EC2 > Target Groups.

  2. Click Create target group.

    The Step 1 - Specify group details tab appears.

  3. Enter/select the following:

    • Target type: IP addresses

    • Target group name: A name of your choice

    • Protocol/Port: The protocol and target port . For example: UDP / 30514

    • VPC: The VPC you have defined previously

    • IP Address Types: IPv4

    • Health checks: TCP

  4. Click Next.

    The Step 2 - Register Targets tab appears.

  5. Enter details:

    • IPv4 address: The IP address of the instance created previously

    • Ports: The target port you entered above.

  6. Click Include as pending below.

  7. Click Create target group.

Create a Load Balancer

The load balancer you create is going to have listeners - one for each of the target group ports from the previous section.

  1. In your AWS console, navigate to EC2 > Load Balancers.

  2. Click Create Load Balancer.

  3. Click Create for Network Load Balancer.

  4. Enter/select details:

    • Load balancer name: A name of your choice

    • Scheme: Internal

    • VPC: The VPC you are using with the instance.

  5. Select the relevant availability zones and subnets you are using.

  6. Add a listener for each target port.

    To add a listener,

    1. Enter/select:

      • Protocol: Protocol. For example: UDP

      • Port: Source port. For example: 514

      • Target group: Name of the appropriate group created in Create Target Groups.

    2. Click Add listener.

  7. Click Create load balancer.

    The load balancer will be added to the list of load balancers

  8. Select the newly created load balancer from the list of load balancers and note the DNS name. This will be the URL of TOS Aurora when it is installed.

Partition the Disk

  1. Log in to the instance as root.

  2. Run the fdisk utility to list the details of all attached disks:

    [<ADMIN> ~]# fdisk -l
    fdisk -l

  3. Run the fdisk utiliy on the your selected disk to partition it:

    [<ADMIN> ~]# fdisk <diskpath>
    fdisk <diskpath>

    where <diskpath> is the disk name preceded by /dev/ as it appears in the output above.

    Example:

    # fdisk /dev/nvme1n1

  4. When prompted, enter as follows:

    Command: n (new partition)

    Partition type: p (primary)

    Partition number: Leave blank for default

    First sector: Leave blank for default

    Last sector: Leave blank for default

    Command: w (write)

    Example input/output:

    [root@ip-172-25-74-123 ~]# fdisk /dev/nvme1n1
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xda95f60b.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-629145599, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-629145599, default 629145599):
Using default value 629145599
Partition 1 of type Linux and of size 300 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

    The partition is created and the fdisk utility will exit.

  5. Format the newly partitioned disk:

    [<ADMIN> ~]# mkfs.xfs <partitionpath>
    mkfs.xfs <partitionpath>

    where <partitionpath> is the partition name preceded by /dev/ as it appears in the output from step 5 above.

    Example:

    # mkfs.xfs /dev/nvme1n1p1

  6. Mount the partition to directory /opt :

    [<ADMIN> ~]# mount <partitionpath> /opt
    mount <partitionpath> /opt

    where <partitionpath> is the partition name preceded by /dev/ as it appears in the output from step 5 above.

    Example:

    # mount /dev/nvme1n1p1 /opt

  7. Configure the system to mount the partition to directory /opt on every boot.

    1. Edit file /etc/fstab.

    2. Add a line to the file:

      <partitionpath> /opt xfs defaults 0 0

      where <partitionpath> is the partition name preceded by /dev/ as it appears in the output from step 5 above.

      Example

      /dev/nvme1n1p1 /opt xfs defaults 0 0

    3. Save the file.

  8. Repeat the steps above for the other partitions specified in Prerequisites.

Configure RHEL/CentOS

  1. If you are not currently logged in as user root, do so now.

    [<ADMIN> ~]$ su -
    su -

  2. If you want to change the host name or IP of the machine, do so now. Once TOS Aurora has been installed, changing the host name or IP address will require reinstalling - see Changing IP Address/Host Names. To change the host name, use the command below, replacing <mynode> with your preferred name.

    [<ADMIN> ~]# hostnamectl set-hostname <mynode>
    hostnamectl set-hostname <mynode>

  3. Modify the environment path to run TOS CLI commands without specifying the full path (/usr/local/bin/tos). 

    [<ADMIN> ~]# echo 'export PATH="${PATH}:/usr/local/bin"' | sudo tee -a /root/.bashrc > /dev/null
    echo 'export PATH="${PATH}:/usr/local/bin"' | sudo tee -a /root/.bashrc > /dev/null

  4. Synchronize your machine time with a trusted NTP server. Follow the steps in Configuring NTP Using Chrony.

  5. Configure the server timezone.

    [<ADMIN> ~]# timedatectl set-timezone <timezone>
    timedatectl set-timezone <timezone>

    where <timezone> is in the format Area/Location. Examples: America/Jamaica, Hongkong, GMT, Europe/Prague. List the time-zone formats that can be used in the command.

    [<ADMIN> ~]# timedatectl list-timezones
    timedatectl list-timezones

  6. Upgrade the kernel:

    [<ADMIN> ~]# dnf upgrade
    dnf upgrade

  7. Disable SELinux:

    • If file /etc/selinux/config exists, edit and change the value of SELINUX to disabled:

      SELINUX=disabled
    • If the file doesn't exist or SELINUX is already set to disabled, do nothing.
  8. Reboot the machine and log in.
  9. Install Wireguard. This is needed to encrypt communication between nodes (machines) within the cluster. See Install Wireguard and follow the steps for your Linux distribution.
  10. Reboot the machine and log in.

  11. Install screen and rsync:

    [<ADMIN> ~]# dnf install -y rsync screen
    dnf install -y rsync screen

  12. Disable the firewall:

    [<ADMIN> ~]# systemctl stop firewalld
    systemctl stop firewalld
    [<ADMIN> ~]# systemctl disable firewalld
    systemctl disable firewalld

  13. Create the TOS Aurora load module configuration file /etc/modules-load.d/tufin.conf. Example using vi:

    [<ADMIN> ~]# vi /etc/modules-load.d/tufin.conf
    vi /etc/modules-load.d/tufin.conf

  14. Specify the modules to be loaded by adding the following lines to the configuration file created in the previous step. The modules will then be loaded automatically on boot.

    br_netfilter
    wireguard
    overlay
    ebtables
    ebtable_filter
    br_netfilter wireguard overlay ebtables ebtable_filter

  15. Load the above modules now:

    [<ADMIN> ~]# cat /etc/modules-load.d/tufin.conf |xargs modprobe -a 
    cat /etc/modules-load.d/tufin.conf |xargs modprobe -a

    Look carefully at the output to confirm all modules loaded correctly; an error message will be issued for any modules that failed to load.

  16. Check that Wireguard has loaded correctly. 

    [<ADMIN> ~]# lsmod |grep wireguard
    lsmod |grep wireguard

    The output will appear something like this:

    wireguard              201106  0
ip6_udp_tunnel         12755  1 wireguard
udp_tunnel             14423  1 wireguard

    If Wireguard is not listed in the output, contact support.

  17. Create the TOS Aurora kernel configuration file /etc/sysctl.d/tufin.conf. Example using vi:

    [<ADMIN> ~]# vi /etc/sysctl.d/tufin.conf
    vi /etc/sysctl.d/tufin.conf

  18. Specify the kernel settings to be made by adding the following lines to the configuration file created in the previous step. The settings will then be applied on boot. 

    net.bridge.bridge-nf-call-iptables = 1
    fs.inotify.max_user_watches = 1048576
    fs.inotify.max_user_instances = 10000
    net.ipv4.ip_forward = 1
    net.bridge.bridge-nf-call-iptables = 1 fs.inotify.max_user_watches = 1048576 fs.inotify.max_user_instances = 10000 net.ipv4.ip_forward = 1

  19. Apply the above kernel settings now:

    [<ADMIN> ~]# sysctl --system
    sysctl --system
For maximum security, we recommend only installing official security updates and security patches for your Linux distribution, as well as the RPMs specifically mentioned in this section.

Create Partitions

You will need to configure three partitions: /opt, /tmp and /var, as well as a separate disk for etcd. The /opt partition will contain your data, which will increase over time. Most of your available disk space should be allocated to this partition and the minimum is determined by the load model parameter (small, medium, large) provided by your account team. Minimum sizes for all partitions:

Minimum Partition Sizes

/opt/

(Small)*

/opt/

(Medium)*

/opt/

(Large)*

/tmp/

 

/var/

 

etcd
Central cluster / remote cluster primary data node / HA data nodes 80 GB 170 GB 370 GB 25 GB 200GB 128 GB
Worker node (central and remote clusters) 70 GB 70 GB 70 GB 25 GB 60 GB N/A

*Small, medium and large refer to the load model parameter provided by your account team.

We recommend allocating /opt partition all remaining disk space after you have partitioned the other directories.

Can I Proceed?

Continue to the next step only if...

  • You have completed the setup described above.