Manual Chapter : Setting up SSL Orchestrator in a High Availability Environment

Applies To:

Show Versions Show Versions

F5 SSL Orchestrator

  • 13.0.0
Manual Chapter

Overview: Setting up SSL Orchestrator in a high availability environment

This section describes how to deploy SSL Orchestrator High Availability (HA). SSL Orchestrator HA configuration and deployment ensures a decrease in downtime and eliminates single points of failure. The deployment of SSL Orchestrator’s HA works in concert with the BIG-IP® device groups support to sync the SSL Orchestrator specific configuration items and is transparent to the user. The deployment occurs after completing a configuration change and selecting Deploy. The deploy request is first routed to one of the devices in the HA device group. This first device configures the box where the request is received. After successful deployment on that box, the request is repeated on other BIG-IP devices. With SSL Orchestrator installed onto a dedicated system with failover, it automatically takes over in case of system failure. Data is synchronized between the two systems ensuring high availability and consistent protection.
Note: SSL Orchestrator high availability deployment is only supported for use with the SSL Orchestrator iApp 2.1 version.

Assumptions and dependencies

  • HA Setup: BIG-IP® HA (CMI) must be set to Active-Standby mode with network failover. See the BIG-IP Device Service Clustering: Administration document for detailed information on Active-Standby HA mode.
  • HA Setup: If the deployed device group is not properly synced or .rpm packages are not properly syncing, make sure your HA self IP (for example, ha_self) Port Lockdown is not set to Allow None. On the Main tab, click Network > Self IPs and click on your ha_self. If Port Lockdown is set to Allow Custom, check that the HA network port 443 is open on self IP.
  • BIG-IP HA Devices: Only manual sync is supported.
  • BIG-IP HA Devices: For use with SSL Orchestrator iApp 2.1, the devices in each BIG-IP HA pair must be the same model and run the same version of TMOS (including any hotfixes). Except for the management interface, you must configure both devices to use the same arrangement of network interfaces, trunks, VLANs, self IPs (address and subnet mask), and routes. For example, if one BIG-IP is connected to a specific VLAN/subnet via interface 1.1, the other BIG-IP must also be connected to that VLAN/subnet via interface 1.1. If the BIG-IP configurations do not match, this solution will not deploy correctly and HA failover will not work.
  • User Experience: False positive configuration conflicts in the HA environment must be ignored.
  • User Experience: Deployment must be initiated from Active HA BIG-IP.
  • User Experience: If a non HA environment is changed to an HA environment, the application must be redeployed. Similarly, if an HA environment is changed to a non HA environment, the application must be redeployed.
  • User Experience: The SSL Configuration page ( SSL Orchestrator > Configuration ) for each peer device can be refreshed in order to see all modified changes.

SSL Orchestrator high availability deployment

To ensure that your SSL Orchestrator HA deployment succeeds, it is critical that each deployment step, as well as the assumptions and dependencies, are closely followed for both boxes. In addition, adhere to all prerequisites, noting that if the systems in the device group are not configured consistently, the deployment synchronization process may fail.

  • Installing an updated .rpm file
  • Configuring the network for high availability
    • Configuring the ConfigSync and Failover IP address
    • Adding a device to the local trust domain
    • Creating a Sync-Failover device group
  • Synchronizing the device group
  • Setting up a basic configuration for deployment

It is also critical to test the deployment after configuration as some failures may not be reported in the UI.

Prerequisites

  • You have made sure that the information used to configure your devices are identical on both boxes before configuring the network for high availability. Without identical information on both devices, noted throughout the following steps, the HA deployment may fail.
  • You must have successfully setup a HA ConfigSync device group prior to starting configuration. Basic instructions are below. For more detailed instructions, refer to the BIG-IP Device Service Clustering: Administration document, section "Managing Configuration Synchronization".
  • You have successfully installed the most current .rpm file on the first device (the Active device).
  • You have already installed SSL Orchestrator with the appropriate license information using the SSL Orchestrator Setup Wizard (or the CLI) and made sure your device setup information is identical on both boxes:
    • While using the SSL Orchestrator Setup Wizard, you have noted the details used for NTP and DNS setup and made sure they will be identical on both boxes. You may verify duplication by selecting System > Configuration > Device > NTP (or DNS).
    • You have made sure that any certificates used in the configuration are copied to all devices.
    • You have made sure that information is identical on all devices. This information should include any of the following that are needed:
      • Client network
      • External network
      • Decrypt zone network
      • Decrypt zone control network
      • Networks providing access to ICAP devices and Receive-only devices.
    • You have made sure the log publishers are configured and named the same.
    • You have made sure all systems use the same interfaces for any services (if interface 1.1 is used to send traffic to an inline layer 2 device on system A, interface 1.1 must also be used on systems B, C, and D).
    Note: Do not attempt to duplicate the configuration by saving and restoring a user configuration set (UCS) file from one machine to the other or any other cloning approach. There are several IDs that are required to be unique that will also be duplicated, causing additional problems.
    Note: For more detailed information on using the SSL Orchestrator Setup Wizard, see the "Using the SSL Orchestrator setup wizard" section.

Installing an updated .rpm file

Make sure you have the latest version of SSL Orchestrator. This will establish the version that will later appear on your other BIG-IP® HA peer device. After downloading the latest version of the SSL Orchestrator zip file from downloads.F5.com, return to your SSL Orchestrator configuration utility. See the section Update the SSL Orchestrator version for more detailed installation instructions.

  1. Create a backup of your current configuration.
  2. On the Main tab, click SSL Orchestrator > Updates .
  3. In the File Name field, click Browse and navigate to the file you saved onto your system.
  4. Click Open to select it.
  5. Click Install.
    Note: Only install the iApp package (the *.rpg file) on the Active system. That system will copy it to the other systems in the ConfigSync group.

Later, after a successful SSL Orchestrator HA deployment, you should verify that the same version appears on the BIG-IP HA peer device.

Configuring the network for high availability

On the Active device, specify the settings for VLAN HA and self IP addresses. If needed, configure all devices involved in the high availability group for HA.

Note: This network will connect the various devices and must be a common layer-2 network between all devices.
  1. On the Main tab, click Network > VLANs . The VLAN List page appears.
  2. Click Create. A new page appears to configure your new VLAN.
  3. In the Name field of General Properties, enter the name (for example, ha_vlan).
  4. For the Interfaces setting:
    1. From the Interfaces list, select an interface number.
    2. From the Tagging list, select Tagged for traffic, for that interface, to be tagged with a VLAN ID.
    3. Click Add. The interface number you selected will appear as a tagged service.
    4. Select Finished.
    5. Next to the F5 logo, your device status will appear showing ONLINE (ACTIVE) and Standalone with green indicators showing their status as up and running.
  5. On the Main tab, click Network > Self IPs . The New Self IP Configuration page appears.
  6. In the Name field, enter the self IP name (for example, ha_self).
  7. In the IP Address field, enter the IP address for the device.
  8. In the Netmask field, enter the netmask for the device.
  9. In the VLAN/Tunnel field, select the VLAN name (ha_vlan).
  10. Click Finished.

Configuring ConfigSync and failover IP addresses

Before creating the device group, you should configure the configuration synchronization (ConfigSync) and Failover IP addresses for each BIG-IP® system in the device group. The ConfigSync address is the IP address that the system uses when synchronizing configuration with peer devices, and the Failover address is the IP address that the system uses for network failover.

  1. On the Main tab, click Device Management > Devices . The Device List page appears with your current device showing in the list.
  2. Click on your device in the device list. The device Properties page appears.
  3. Select the ConfigSync tab. The ConfigSync Configuration section appears showing the Local Address of that device.
  4. From the Local Address list, select the VLAN address (ha_vlan).
  5. Click Update.
  6. Select the Failover Network tab and click Add. The New Failover Unicast Address page opens.
    In the Address field, make sure that the VLAN address (ha_vlan) is present.
  7. Click Repeat.
  8. After the page refreshes, from the Address list, select the Management Address.
    Note: Connection Mirroring is not supported.
  9. Click Finished. The Failover Unicast Configuration section will list both the VLAN HA (ha_valn) and Management Address devices.

Adding a device to local trust domain

Any BIG-IP® devices that you intend to add to a device group must first be members of the same local trust domain. When a BIG-IP device joins the local trust domain, it establishes a trust relationship with peer BIG-IP devices that are members of the same trust domain. For example, if you are creating a device group with two members, you must log in to one of the devices and join the other device to that system's local trust domain. The devices can then exchange their device properties and device connectivity information.

  1. On the Main tab, click Device Management > Device Trust . The Local Domain page appears.
  2. Select the Device Trust Members tab. The Peer and Subordinate Devices page appears.
  3. Click Add. The Device Trust page appears with the Retrieve Device Credentials (Step 1 of 3) section.
  4. In the Device Type field, select Peer.
  5. In the Device IP Address field, enter the IP address of your device.
  6. Click Retrieve Device Information. The Verify Device Certificates (Step 2 of 3) section appears.
  7. Click Device Certificate Matches. The Add Device (Step 3 of 3) section appears.
  8. In the Name field, enter the name of the device you are adding.
  9. Click Add Device. Next to the F5 logo, the status of your device should show ONLINE (ACTIVE) and Connected with a green indicator next to it showing its active and connected status.

Creating a sync-failover device group

This task establishes failover capability between two or more BIG-IP® devices. If an active device in a Sync-Failover device group becomes unavailable, the configuration objects fail over to another member of the device group and traffic processing is unaffected. You perform this task on any one of the authority devices within the local trust domain.

  1. On the Main tab, click Device Management > Device Groups . The New Device Group page appears.
  2. Click Create.
  3. In the General Properties section, do the following:
    1. In the Name field, enter the name of your device group.
    2. In the Group Type field, select Sync-Failover from the list.
  4. In the Configuration section, do the following:
    1. In the Members field, select both available devices from the Available list and add them to the Includes list.
    2. In the Sync Type field, select Manual with Incremental Sync.
      Note: You must do a manual sync. If you select Automatic with Incremental Sync, your HA deployment will fail.
  5. Click Finished.
The Device Group List page appears listing your new device group. The ConfigSync Status column will indicate Awaiting Initial Sync.

Synchronizing the device group

This task synchronizes the BIG-IP® configuration data from the local device to the devices in the device group. This synchronization ensures that devices in the device group operate properly. When synchronizing self IP addresses, the BIG-IP system synchronizes floating self IP addresses only.

  1. Next to the F5 logo, click on Awaiting Initial Sync. The Device Management Overview page appears showing your Device Groups.
  2. In the Sync Issues section, select ha to expand the Devices and Sync Options sections.
  3. In the Devices section, make sure you select the device showing Changes Pending.
  4. In the Sync Options section, select Push the selected device configuration to the group.
  5. Click Sync.

You have now completed your SSL Orchestrator high availability deployment. Next, setup a basic configuration for deployment on your Active device.

Setting up a basic configuration for deployment

Refer to the "Setting Up a Basic Configuration" section for detailed instructions on completing the basic configuration on your Active device.

Note: You must create identical information on each device before deploying the configuration.

After deploying your configuration on the Active device, the configuration is automatically synchronized with all of the other devices in the device group. Since some errors may not be apparent, it is critical that you thoroughly test and diagnose the success or failure of the deployment. The following steps can be taken to test the system.

Task summary for diagnosing and fixing high availability deployment

Even though the potential for SSL Orchestrator HA deployment is low, thorough verification is recommended. If your HA deployment fails, attempt:

  • Verifying deployment and viewing logs
  • Verifying the .rpm file version on both devices
  • Configuring general properties and redeploying
  • Reviewing error logs and performing recovery steps

Verifying deployment and viewing logs

Verify that all expected and required virtuals, profiles, and BIG-IP® LTM and network objects (route-domains, VLANs, self IPs) have been created on each device in the HA device group. These will be items beginning with the name given to the application (for example, if the application was named SSLO, verify that all of the items named | Summary SSL Orchestrator 13.0.0 | 9 SSLO_* are the same on all boxes). Ensure that the .rpm files are in sync, verify deployment with or without services, and review the following logs for failures:

  • /var/log/restnoded/restnoded.log
  • /var/log/restjavad.0.log
Note: Because the initial device in the HA device group repeats the configuration requests and propagates the configuration to other BIG-IP devices, make sure you verify the initial configured device first, followed by each device in the HA device group. If the initial device deployment configuration fails, all other device configuration deployments will not successfully be configured.

Verifying the .rpm file version on both devices

After a successful SSL Orchestrator HA deployment, verify that the latest version of the SSL Orchestrator zip file is installed on both devices.

  1. On the Main tab, click SSL Orchestrator > Updates .
  2. Check the versions in the Version field.

If the versions are not identical, you must install an updated .rpm file and verify that both boxes are identically configured.

Configuring general properties and redeploying

  1. Remove all configurations present on all devices.
    Note: You may want to restore a backup file instead, per device, to remove all current configurations.
  2. For all devices, individually configure each section in the iApp and select Deploy. Verify that all new objects are properly synced and deployed.
    Note: If synchronization or deployment issues persist after deploying after each section, attempt to deploy after updating each item (instead of after each section) in the iApp and verify that all new objects are properly synced and deployed.
    Note: See the "Configuring general properties" section for more detailed information.

Reviewing error logs and performing recovery steps

  1. Verify that all BIG-IP® LTM and network objects are present on each of the devices in the HA device group.
  2. If the configuration deployment fails on each device, review the logs:
    • /var/log/restnoded/restnoded.log
    • /var/log/restjavad.0.log
  3. Use the following REST GET command to determine the state of the deployed device block in the REST storage:
    • curl -s -k -u admin:admin https://localhost/mgmt/shared/iapp/blocks | json-format
  4. Since failure scenarios can vary, after reviewing the logs, attempt the following recovery steps:
    1. Redeploy SSL Orchestrator.
      If this succeeds, you have recovered from the failure situation.
    2. Undeploy SSL Orchestrator.
      By undeploying, a cleanup of MCP objects on each of the boxes occurs while also cleaning up required data properties within the block stored in REST storage. If this succeeds, attempt to redeploy again.
    3. If redeploy or undeploy fails, do the following:
      1. From command line (back door), run > touch /var/config/rest/iapps/enable.
      2. Refresh the SSL Orchestrator menu UI.
      3. Select the deployed application from the list and delete the application.
      4. Redeploy and undeploy again.
      5. Once done, remove the file rm -f /var/config/rest/iapps/enable.
    4. If these recovery steps do not work, you may need to clean up the REST storage.
Note: For more detailed information on setting up HA, see the BIG-IP Device Service Clustering: Administration document.