Manual Chapter : Planning and Implementing a Centralized Management Deployment

Applies To:

Show Versions Show Versions

BIG-IQ Centralized Management

  • 5.2.0
Manual Chapter

What elements make up a Centralized Management deployment?

A F5® BIG-IQ® Centralized Management solution involves a number of components. This diagram illustrates those that make up the BIG-IQ® Centralized Management solution.

Centralized Management network topology

BIG-IQ Centralized Management device

F5® BIG-IQ® Centralized Management is a platform that helps you manage BIG-IP® devices, and all of their services (such as LTM®, AFM™, or ASM®), from one location. That means you and your co-workers don't have to log in to individual BIG-IP systems to get your job done. Using BIG-IQ Centralized Management, you can centrally manage your BIG-IP devices, performing operations such as backups, licensing, monitoring, and configuration management. And because access to each area of BIG-IQ is role based, you can limit access to users maximizing work flows while minimizing errors and potential security issues.

BIG-IQ data collection device

A BIG-IQ data collection device (DCD) is a specially-provisioned BIG-IQ® system that you use to manage and store alerts, events, and statistical data from one or more BIG-IP® systems.

Configuration tasks on the BIG-IP system determine when and how alerts or events are triggered on the client. The alerts or events are sent to a BIG-IQ data collection device, and the BIG-IQ system retrieves them for your analysis. When you opt to collect statistical data from the BIG-IP devices, the DCD periodically retrieves those statistics from your devices, and then processes and stores that data.

The group of data collection devices and BIG-IQ systems that work together to store and manage your data are referred to as the data collection cluster. The individual data collection devices are generally referred to as nodes.

BIG-IP device

A BIG-IP device runs a number of licensed components designed around application availability, access control, and security solutions. These components run on top ofF5® TMOS®. This custom operating system is an event driven operating system designed specifically to inspect network and application traffic and make real-time decisions based on the configurations you provide. The BIG-IP software runs on both hardware and virtualized environments.

Remote storage device

The remote storage device is necessary only when your deployment includes a data collection device (DCD) and you plan to store backups of your events, alerts, and statistical data for disaster recovery requirements.

Network Requirements for a BIG-IQ Centralized Management Deployment

Before you deploy a Centralized Management solution

Before you begin to deploy a BIG-IQ® system, you should complete these preparations.

  • Determine the deployment scenario that works best for your needs.
  • Create the interfaces, communications, and networks needed to support your deployment scenario
  • Configure your network (including switches and firewalls) to permit BIG-IQ network traffic to flow based on the deployment scenario you choose.
  • Assemble the passwords, IP addresses, and licensing information needed for the BIG-IQ cluster components.

Planning for a Centralized Management deployment

To successfully deploy a BIG-IQ® Centralized Management solution, you may need to coordinate with several people in your company.

If you use BIG-IQ virtual editions, you might need to coordinate with the people who manage your virtual environment, so they can provision the virtual machines with the required amount of CPUs, memory, and network interfaces. Further, you’ll need to coordinate with the people who manage the storage for the virtual machines to make sure each virtual machine is provisioned with the necessary storage to support the BIG-IQ environment. You also might need to provide the virtual environment team a copy of the BIG-IQ virtual machine image (available from https://downloads.f5.com), depending how they operate.

If you use BIG-IQ 7000 devices in your network, you need to coordinate with the people who manage the data center where the BIG-IQ devices are housed to make arrangements for the devices to be racked, powered on, and connected to your network.

There are also several tasks to coordinate with your networking team:
  • IP address allocation for the BIG-IQ nodes, depending on your deployment model.
  • Creation of networks, VLANs, and so on dependent on your deployment model.
  • Any routing configuration required to ensure traffic passes between the BIG-IQ nodes and the BIG-IP devices.
  • Additional networking configuration required to support the BIG-IQ system's operation.

Finally, you may need to coordinate with your network firewall administrators, depending on the network configuration at your company. The BIG-IQ software needs to communicate between BIG-IQ nodes and BIG-IP systems; and, if there are firewalls in the network path, firewall rules probably need to be configured to permit that traffic. For additional detail about required network ports and protocols, refer to Open ports required for data collection device cluster deployment.

Determining the network configuration needed for your deployment

There are three common deployment scenarios for the BIG-IQ® system. The scenario most appropriate for you depends on what you want to do.
Table 1. BIG-IQ deployment options
Which deployment type should you choose? What functions does your deployment need to perform? Which hardware components and networks do you need?
Simple management and configuration. Manage and configure the BIG-IP® devices. For example, taking backups, licensing virtual editions, and configuring local traffic and security policies. All you need is one or more BIG-IQ system and the BIG-IP devices you want to manage. This configuration uses a single management network.
Advanced management and configuration. Manage and configure the BIG-IP devices. For example, taking backups, licensing virtual editions, and configuring local traffic and security policies.

Collect and view Local Traffic, DNS, and Device statistical data from the BIG-IP devices.

Collect, manage, and view events and alerts from BIG-IP devices provisioned with the APM®, FPS, or ASM® components.

You need BIG-IQ systems, data collection devices, and an external storage device. This configuration needs a single, management network and an internal BIG-IQ cluster network.
Large-scale, distributed management and configuration Manage and configure the BIG-IP devices. For example, taking backups, licensing virtual editions, and configuring local traffic and security policies.

Collect and view Local Traffic, DNS, and Device statistical data from the BIG-IP devices.

Collect, manage, and view events and alerts from BIG-IP devices provisioned with the APM, FPS, or ASM components.

Separate network traffic to support large, distributed deployments of the F5 BIG-IQ Centralized Management solution for improved performance, security, and interactions in multiple data center environments.

You need BIG-IQ systems, data collection devices, and an external storage device. This configuration needs an external network, a management network, and an internal network.
Network environment for simple management and configuration

To deploy this configuration, all you need is one or more BIG-IQ® systems and the BIG-IP® devices you want to manage. The number of BIG-IQ systems you need depends on how much redundancy your business requires. A second system provides high availability failover capability. Or, for disaster recovery capability, you could operate multiple data centers, each with its own set of BIG-IQ systems. (For additional detail, refer to Managing Disaster Recovery Scenarios.)

The simple management and configuration uses a single management network. Traffic on the management network is used to do the following:

  • Provide communication between the BIG-IQ system and DCD nodes.
  • Enable bidirectional traffic between the BIG-IQ systems and the BIG-IP devices.
  • If you use a secondary high availability BIG-IQ system, enable traffic between the BIG-IQ systems. This traffic keeps the state information synchronized on your BIG-IQ systems.
  • Provide access the BIG-IQ user interface. You can also use it to access the BIG-IQ system using SSH if you need to run manual commands.
Note: The number of devices of each type that will best meet your company's needs depends on a number of factors. Refer to the Data collection device sizing guidelines for additional detail.

The figure illustrates the network topology required for a simple management and configuration deployment.

Centralized management network topology

Use the table to record the IP addresses for the devices in the BIG-IQ deployment.

Device Type Management IP address(es)
Primary BIG-IQ system  
Secondary BIG-IQ system  
BIG-IP devices  
Network environment for advanced management and configuration

To deploy this configuration, you need BIG-IQ® systems, Data Collection Devices, and an external storage device. This configuration needs a single management network, an internal BIG-IQ cluster network, and an optional external storage device for backing up alert, event, and statistical data..

Note: The number of devices of each type that will best meet your company's needs depends on a number of factors. Refer to the Data collection device sizing guidelines for additional detail.

Traffic on the management network is used to do the following:

  • Provide communication between the BIG-IQ system and DCD nodes.
  • Enable bidirectional traffic between the BIG-IQ systems and the BIG-IP devices.
  • If you use a secondary high availability BIG-IQ system, enable traffic between the BIG-IQ systems. This traffic keeps the state information synchronized on your BIG-IQ systems.
  • Provide access the BIG-IQ user interface. You can also use it to access the BIG-IQ system using SSH if you need to run manual commands.
The internal network is used to replicate data to maintain the BIG-IQ Centralized Management cluster.
Note: It is best practice to isolate the traffic between BIG-IQ cluster nodes for performance and improved security.

This figure illustrates the network topology required for an advanced management and configuration deployment.

Centralized management and enhanced monitoring network topology

Use the table to record the IP addresses for the devices in the BIG-IQ deployment.

Device Type Management IP addresses Internal Network IP addresses
Primary BIG-IQ system    
Secondary BIG-IQ system    
Data Collection Device management IP addresses    
BIG-IP devices    
Remote storage device    
Network environment for large-scale, distributed management and configuration

To deploy this configuration, you need BIG-IQ® systems, Data Collection Devices, and an external storage device. This configuration needs an external network, a management network, an internal network, and an optional external storage device for backing up alert, event, and statistical data.

The external network routes traffic between the BIG-IQ Centralized Management cluster and the BIG-IP® devices.

The internal network is used to replicate data to maintain the BIG-IQ Centralized Management cluster.
Note: It is best practice to isolate the traffic between BIG-IQ cluster nodes for performance and improved security.

Traffic on the management network is used to do the following:

  • Provide communication between the BIG-IQ system and DCD nodes.
  • Enable bidirectional traffic between the BIG-IQ systems and the BIG-IP devices.
  • If you use a secondary high availability BIG-IQ system, enable traffic between the BIG-IQ systems. This traffic keeps the state information synchronized on your BIG-IQ systems.
  • Provide access the BIG-IQ user interface. You can also use it to access the BIG-IQ system using SSH if you need to run manual commands.

This figure illustrates the network topology required for this deployment.

Centralized management, enhanced monitoring, and improved performance network topology

Use the table to record the IP addresses for the devices in the BIG-IQ deployment.

Device Type Management IP addresses Internal Network IP addresses External Network IP addresses
Primary BIG-IQ system      
Secondary BIG-IQ system      
Data Collection Device management IP addresses      
BIG-IP devices      
Remote storage device      

Open ports required for data collection device cluster deployment

The BIG-IQ® system must have bidirectional communication with the devices in your network to successfully manage them. The ports described in the table must be open to allow for this required two-way communication. You might have to contact a firewall or network administrator to verify that these ports are open, or to have them opened if they are not.

Table 2. Ports required for data collection device cluster deployment
Source IP Address Destination IP Address Destination Port Protocol Is port Configurable? Is the Protocol Configurable? Purpose Connection Origination
Management IP address or external self IP address of the BIG-IQ console.

*See table note 1.

Management IP address or self IP address of the BIG-IP device.

*See table note 1.

443 (SSL)

22 (SSH)

*See table note 4.

TCP No No Device-level discovery, device configuration changes, and device operations (backup, licensing, and so on), health checking, and some statistics (For example, Access or ADC object status). From BIG-IQ console to BIG-IP devices.
Management IP address or external self IP address of the BIG-IQ data collection device.

*See table notes 1 and 2.

Management IP address or self IP address of the BIG-IP device.

*See table note 1.

443 (SSL) TCP No No Statistics collection for Local Traffic, Device, and DNS objects. From BIG-IQ data collection devices to BIG-IP devices.
Management IP address or internal self IP address of the BIG-IQ console.

*See table note 1.

Management IP address or internal self IP address of the BIG-IQ console.

*See table note 1.

443 (SSL) TCP No No BIG-IQ cluster synchronization and cluster maintenance.

From the active BIG-IQ console to the standby BIG-IQ console.

From the BIG-IQ standby console to the BIG-IQ active console.

Management IP address or internal self IP address of the active BIG-IQ console.

*See table note 1.

Management IP address or internal self IP address of the standby BIG-IQ console.

*See table note 1.

27017 TCP No No BIG-IQ high availability cluster data replication.

From the active BIG-IQ console to the standby BIG-IQ console.

From the BIG-IQ standby console to the BIG-IQ active console.
Management IP address or internal self IP address of the BIG-IQ console and the data collection device.

*See table notes 1 and 2.

Management IP address or internal self IP address of the BIG-IQ console and the data collection device.

*See table notes 1 and 3.

9300 TCP Yes No Internal node-to-node communication to maintain data consistency and replication across clusters when data collection nodes are used. Full Mesh That is, all BIG-IQ console and data collection devices can originate a connection for this purpose.
Management IP address or self IP address of the BIG-IP device.

*See table note 1.

Management IP address or self IP address of the BIG-IQ data collection device.

*See table notes 1 and 3.

8514 TCP No No Logging profile communication for Web Application Security. This traffic uses the syslog protocol documented in RFC 5424. From BIG-IP devices to BIG-IQ data collection devices. When you have multiple data collection devices, you need to make sure data can pass to all devices in the cluster.
Management IP address or self IP address of the BIG-IP device.

*See table note 1.

Management IP address or self IP address of the BIG-IQ data collection device.

*See table notes 1 and 3.

8008 TCP No No Logging profile communication for Fraud Protection Service, this traffic uses the syslog protocol documented in RFC 5424. From BIG-IP devices to BIG-IQ data collection devices. When you have multiple data collection devices, you need to make sure data can pass to all devices in the cluster.
Management IP address or self IP address of the BIG-IP device. Management IP address or self IP address of the BIG-IQ data collection device.

*See table notes 1 and 3.

9997 TCP No No For access to events; this traffic uses the syslog protocol documented in RFC 5424. From BIG-IP devices to BIG-IQ data collection devices. When you have multiple data collection devices, you need to make sure data can pass to all devices in the cluster.
Client IP address BIG-IQ Management IP address or self IP address for all BIG-IQ instances in the cluster.

443 (SSL)

22 (SSH)

TCP No No For management access to BIG-IQ GUI or API (port 443) or shell access to BIG-IQ (port 22) From the client workstation to the BIG-IQ device.
Note: 1: Whether you use the management IP address or the self IP address depends on your network configuration.
Note: 2: For clusters with multiple data collection devices, traffic must be able to originate from any device in the cluster.
Note: 3: For clusters with multiple data collection devices, the destination can be any device in the cluster.
Note: Port 22 (SSH) is only required for BIG-IP® versions 11.5.0 to 11.6.0

Passwords required for data collection device cluster deployment

To install and configure a data collection device cluster, you use the default passwords for all of the devices in your cluster. If (as recommended) you intend to schedule regular snapshots of your logging data, you need root access credentials for the machine on which you plan to store these snapshots.

Table 3. Passwords for data collection device cluster deployment
User Name Default Password Access Rights/Role
admin admin This user type can access all aspects of the BIG-IQ® system from the system's user interface.
root default This user has access to all aspects of the BIG-IQ system from the system's console command line.

Licenses required for data collection device cluster deployment

To install and configure a data collection device cluster, you need a license for each device.

Data collection device sizing guidelines

This material is being replaced with a new guide scheduled for inclusion in the next release.

Deploying a Data Collection Device

How do I deploy a data collection device cluster?

To manage the data generated by BIG-IP® devices on BIG-IQ® Centralized Management, you deploy a network of devices called a data collection device (DCD) cluster, and then configure that cluster to meet your business needs.

To deploy a DCD cluster, you should:
  • Prepare your network environment
  • Install the DCDs
  • Discover and activate the DCDs
  • Define an external location to store snapshots
  • Enable data collection for the DCD cluster (or configure a BIG-IP system to send alerts or events to the cluster)
  • Configure the BIG-IQ console that manages the DCD cluster for HA, if needed.

License basic setup for a data collection device?

The BIG-IQ® data collection device runs as a virtual machine in supported hypervisors, or on the BIG-IQ 7000 series platform. You license the data collection device using the base registration key you purchased. The base registration key is a character string that the F5 license server uses to provide access to data collection device features.

You license data collection device in one of the following ways:

  • If the system has access to the internet, you can have the data collection device contact the F5 license server and automatically activate the license.
  • If the system is not connected to the internet, you can manually retrieve the activation key from a system that is connected to the internet, and transfer it to the data collection device.
  • If your data collection device is in a closed-circuit network (CCN) that does not allow you to export any encrypted information, you must open a case with F5 support.

When you license the data collection device, you:

  • Specify a host name for the system.
  • Assign a management port IP address.
  • Specify the IP address of your DNS server and the name of the DNS search domain.
  • Specify the IP address of your Network Time Protocol (NTP) servers and select a time zone.
  • Change the administrator’s default admin and root passwords.
Automatically license BIG-IQ and perform initial setup
You must have a base registration key before you can license the BIG-IQ® system. If you do not have a base registration key, contact the F5 Networks sales group (f5.com).
If the BIG-IQ® system is connected to the public internet, you can follow these steps to automatically perform the initial license activation and perform the initial setup.
  1. Use a browser to log in to BIG-IQ by typing https://<management_IP_address> , where <management_IP_address> is the address you specified for device management.
  2. Click Activate.
  3. In the Base Registration Key field, type or paste the BIG-IQ registration key.
    Important: If you are setting up a data collection device, you have to use a registration key that supports a data collection device license.
  4. In the Add-On Keys field, paste any additional license key you have.
  5. To add another additional add-on key, click the + sign and paste the additional key in the new Add-On Keys field.
  6. For the Activation Method setting, select Automatic, and click the Activate button.
  7. In the Hostname field, type a fully-qualified domain name (FQDN) for the system.
    You cannot change this name after you add it. The FQDN can consist of letters and numbers, as well as the characters underscore ( _ ), dash ( - ), or period ( . ).
  8. In the Management Port IP Address and Management Port Route fields, type the IP address for the management port IP address and route.
    Note: The management port IP address must be in Classless Inter-Domain Routing (CIDR) format. For example: 10.10.10.10/24.
  9. Specify what you want the BIG-IQ to use for the Discovery Address.
    • To use the management port, select Use Management Address.
    • To use the internal self IP address, select Self IP Address, and type the IP address.
      Important: If you are configuring a data collection device, you must use the internal self IP address.
      Note: The self IP address must be in Classless Inter-Domain Routing (CIDR) format. For example: 10.10.10.10/24.
  10. In the DNS Lookup Servers field, type the IP address of your DNS server.
    You can click the Test Connection button to verify that BIG-IQ can reach that IP address.
  11. In the DNS Search Domains field, type the name of your search domain.
    The DNS search domain list allows the BIG-IQ system to search for local domain lookups to resolve local host names.
  12. In the Time Servers field, type the IP addresses of your Network Time Protocol (NTP) server.
    You can click the Test Connection button to verify that BIG-IQ can reach the IP address.
  13. From the Time Zone list, select your local time zone.
  14. In the Old Password fields, type the default admin and root passwords, and then type a new password in the Password and Confirm Password fields.
Manually license BIG-IQ and perform initial setup
You must have a base registration key before you can license the BIG-IQ® system. If you do not have a base registration key, contact the F5 Networks sales group (f5.com).
If the BIG-IQ® system is not connected to the public internet, use this procedure to manually activate the license and perform the initial setup.
  1. Use a browser to log in to BIG-IQ by typing https://<management_IP_address> , where <management_IP_address> is the address you specified for device management.
  2. Click Activate.
  3. In the Base Registration Key field, type or paste the BIG-IQ registration key.
    Important: If you are setting up a data collection device, you have to use a registration key that supports a data collection device license.
  4. In the Add-On Keys field, paste any additional license key you have.
  5. For the Activation Method setting, select Manual and click the Generate Dossier button.
    The BIG-IQ system refreshes and displays the dossier in the Device Dossier field.
  6. Select and copy the text displayed in the Device Dossier field.
  7. Click the Access F5 manual activation web portal link.
    The Activate F5 Product site opens.
  8. Into the Enter your dossier field, paste the dossier.
    Alternatively, if you saved the file, click the Choose File button and navigate to it.
    After a pause, the screen displays the license key text.
  9. Click Next.
    If you are setting up this device for the first time, the Accept User Legal Agreement screen opens.
  10. To accept the license agreement, select I have read and agree to the terms of this license, and click Next. button.
    The licensing server creates the license key text.
  11. Copy the license key.
  12. In the License Text field on BIG-IQ, paste the license text.
  13. Click the Activate License button.
  14. In the Hostname field, type a fully-qualified domain name (FQDN) for the system.
    You cannot change this name after you add it. The FQDN can consist of letters and numbers, as well as the characters underscore ( _ ), dash ( - ), or period ( . ).
  15. In the Management Port IP Address and Management Port Route fields, type the IP address for the management port IP address and route.
    Note: The management port IP address must be in Classless Inter-Domain Routing (CIDR) format. For example: 10.10.10.10/24.
  16. Specify what you want the BIG-IQ to use for the Discovery Address.
    • To use the management port, select Use Management Address.
    • To use the internal self IP address, select Self IP Address, and type the IP address.
      Important: If you are configuring a data collection device, you must use the internal self IP address.
      Note: The self IP address must be in Classless Inter-Domain Routing (CIDR) format. For example: 10.10.10.10/24.
  17. In the DNS Lookup Servers field, type the IP address of your DNS server.
    You can click the Test Connection button to verify that BIG-IQ can reach that IP address.
  18. In the DNS Search Domains field, type the name of your search domain.
    The DNS search domain list allows the BIG-IQ system to search for local domain lookups to resolve local host names.
  19. In the Time Servers field, type the IP addresses of your Network Time Protocol (NTP) server.
    You can click the Test Connection button to verify that BIG-IQ can reach the IP address.
  20. From the Time Zone list, select your local time zone.
  21. In the Old Password fields, type the default admin and root passwords, and then type a new password in the Password and Confirm Password fields.

Discover and activate a data collection device

Using BIG-IQ® Centralized Management, you can discover a data collection device, and add it to the Logging Group. The BIG-IQ can then access the data on the discovered data collection device. You can then receive data from multiple BIG-IP® systems. This unified view makes browsing easier, and provides a complete view of application alert or event activity and statistics data.
  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click Add .
  4. On the New BIG-IQ Data Collection Device screen, fill in as appropriate:
    1. In Management Address, type the management IP address.
    2. In Username, type the user name for an administrator on the data collection device (for example, admin.
    3. In Password, type the password for an administrator on the data collection device (for example, admin.
    4. In Data Collection IP Address, type the IP address of the data collection device internal self IP address.
    5. For Data Collection Port, the default value is 9300. The BIG-IQ uses this port for internal polling and communication with the data collection devices.
    6. For Zone, select the zone in which the data collection device resides.
      Your Admin sets up the zones so that the nodes in your cluster are distributed equitably for disaster recovery purposes.
  5. Click the Add button at the bottom of the screen to add the data collection device to the system.
    Note: This operation might take a minute or two.
  6. Repeat the preceding steps for each data collection device you want to configure.
  7. To activate this device for the service you want to monitor, on the BIG-IQ Data Collection Devices screen, in the Services column, click Add Services.
    The Services screen for this data collection device opens.
  8. For the service you want to add, confirm that the Listener Address correctly specifies the external self IP address of the data collection device, and click Activate.
    When the service is successfully added, the Service Status changes to Active.
  9. Click Save & Close.
Once discovered and activated, this data collection device collects the data generated by the configured BIG-IP systems. Thus, BIG-IQ provides a single view of all alert or event entries and statistics data.
Important: The Total Document Count is not a report of the number of alerts or events sent to the data collection device. Instead, it is a sum of various document types sent to the data collection device. Events and alerts are included in this list, but this total includes other document types as well.

Modifying alert log indices for Access

Before you can configure the indices for a data collection device, you must activate services for the components that you want to collect data for.
Alert log indices determine the physical characteristics of what is sent to the data collection device. You can set up index rotation for Access alerts.
Important: The configuration process for log indices varies significantly depending on the component that generates the data. Use the process that corresponds to the component you are trying to set up.
  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click the Settings button.
    The Settings screen opens to display the current state of the DCD cluster defined for this BIG-IQ device.
  4. On the left, click Logging Data Collection .
  5. For Access Policy (APM), click the Configure button.
    The Access Indices screen opens.
  6. Perform the next two steps for each section on this screen.
    Important: To avoid a mismatch in the reports generated from your logging data, use the same indices values for the access-event-logs and access stats.
  7. Specify the Rotation Type.
    • To chunk your data based on the amount of data:
      1. Select Size Based
      2. For the Max Index Size, type the size of the chunks you want event log data sent to the DCD.
      Note: For example, if you type 1000, when the event log data reaches a size of 1 Gig, it will be sent to the DCD.
    • To chunk your data based on the increments of time:
      1. Select Time Based
      2. For the Rotation Period, specify a time unit, and type how many of those you want to comprise each data chunk sent to the DCD.
      Note: For example, if you type .5 and select Hours, the event log data will be sent to the DCD every half hour.
  8. For the Retained Index Count, type the total number of indexes you want to store on the DCD.
    The maximum amount of data stored on the DCD is determined by this setting. When the amount of data reaches this size, the oldest data is truncated or discarded.
  9. Click Save & Close to save the indices configuration settings.

Modifying alert log indices for Web Application Security

Before you can configure the indices for a data collection device, you must activate services for the components that you want to collect data for.
Alert log indices determine the physical characteristics of what is sent to the Data Collection Device. Use this task to set up index rotation for Web Application Services alerts.
Important: The configuration process for log indices varies significantly depending on the component that generates the data. Use the process that corresponds to the component you are trying to set up.
  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click the Settings button.
    The Settings screen opens to display the current state of the DCD cluster defined for this BIG-IQ device.
  4. On the left, click Logging Data Collection .
  5. For Web Application Security (ASM), click the Configure button.
    The ASM Indices screen opens.
  6. Specify the Rotation Type.
    • To chunk your data based on the amount of data:
      1. Select Size Based
      2. For the Max Index Size, type the size of the chunks you want event log data sent to the DCD.
      Note: For example, if you type 1000, when the event log data reaches a size of 1 Gig, it will be sent to the DCD.
    • To chunk your data based on the increments of time:
      1. Select Time Based
      2. For the Rotation Period, specify a time unit, and type how many of those you want to comprise each data chunk sent to the DCD.
      Note: For example, if you type .5 and select Hours, the event log data will be sent to the DCD every half hour.
  7. For the Retained Index Count, type the total number of indexes you want to store on the DCD.
    The maximum amount of data stored on the DCD is determined by this setting. When the amount of data reaches this size, the oldest data is truncated or discarded.
  8. Click Save & Close to save the indices configuration settings.

Modifying event log indices for FPS

Before you can configure the indices for a data collection device, you must activate services for the components that you want to collect data for.
Event log indices determine the physical characteristics of what is sent to the data collection device. You can set up index rotation for Fraud Protection Services events.
Important: The configuration process for log indices varies significantly depending on the component that generates the data. Use the process that corresponds to the component you are trying to set up.
  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click the Settings button.
    The Settings screen opens to display the current state of the DCD cluster defined for this BIG-IQ device.
  4. On the left, click Logging Data Collection.
  5. for Fraud Protection (FPS), click the Configure button .
    The FPS Indices screen opens.
  6. Specify the Rotation Type.
    • To chunk your data based on the amount of data:
      1. Select Size Based
      2. For the Max Index Size, type the size of the chunks you want event log data sent to the DCD.
      Note: For example, if you type 1000, when the event log data reaches a size of 1 Gig, it will be sent to the DCD.
    • To chunk your data based on the increments of time:
      1. Select Time Based
      2. For the Rotation Period, specify a time unit, and type how many of those you want to comprise each data chunk sent to the DCD.
      Note: For example, if you type .5 and select Hours, the event log data will be sent to the DCD every half hour.
  7. For the Retained Index Count, type the total number of indexes you want to store on the DCD.
    The maximum amount of data stored on the DCD is determined by this setting. When the amount of data reaches this size, the oldest data is truncated or discarded.
  8. Click Save & Close to save the indices configuration settings.

Configure secure communications for data collection device

You need a signed SSL certificate before you can configure HTTPS communications to a data collection device.
If you want to secure the communications between the BIG-IP® devices and your data collection device cluster using SSL encryption, you must provide a signed SSL certificate to the BIG-IP devices and F5® BIG-IQ® Centralized Management systems. You do this by configuring both the BIG-IP device and the data collection device.
Note: The BIG-IP device that generates Fraud Protection Service alerts must be configured to send its alerts to the data collection device (DCD). This process is documented in a separate guide. The guide F5 Fraud Protection Service: Configuration, Version 13.0 provides complete setup instructions for using FPS on a BIG-IP system. Complete the standard setup as documented in the guide, except when you configure the alert server pool, add your DCDs to an alerts pool using their internal self IP addresses.
  1. Use SSH to log in to the data collection device.
  2. Replace the content of the /etc/httpd/conf/ssl.crt/ directory on the data collection device with your signed SSL certificate.
  3. Replace the content of the /etc/httpd/conf/ssl.key/ directory on the data collection device with your signed SSL key.
  4. To apply these changes to the data collection device, type: bigstart restart webd and then press Enter.
  5. Log out of the data collection device.

Add a proxy for secure communication

Before you can perform this task, you must be logged in as Admin, and you must have configured a proxy server that your data collection device cluster can access.

As a security precaution, you may want to configure a proxy to route communications. For example you might use it to route your forwarded alerts or download alert rules from the security operations center. Or you might want to use a proxy to avoid exposing the BIG-IQ® device when you download ASM® signature files.
Important: To use a proxy for Fraud Protection Service, you must configure a proxy on each device (each data collection device and both the primary and the secondary BIG-IQ devices) in the cluster. The proxy names you specify for each node in the cluster must match exactly, but the IP address and port number for the proxy can be different from device to device.
  1. At the top of the screen, click System.
  2. On the left, click PROXIES.
  3. On the Proxies screen, click Add.
  4. If you are configuring an HA peer group, from Device, select the primary BIG-IQ system.
  5. For Name, type a name for the proxy you want to use.
    Important: The proxy name must match across all devices in the cluster. The proxy addresses and port can vary.
  6. For Address, type the IP address of the proxy server.
  7. For Port, type the port that you want the proxy server to use.
  8. If the proxy server requires authentication, type the User Name and Password for the proxy.
  9. To add another proxy, click the plus sign in the upper right hand corner, and then repeat the preceding 4 steps.
  10. If your network configuration includes an HA peer, repeat steps 3 - 9, but this time, in step 4, select the HA secondary system.
    Note: The proxy name for the HA secondary must match the name used for the primary. The proxy addresses and port can vary.
  11. Click Save & Close
You need to add a proxy for each data collection device in the cluster.
Note: Remember, the proxy name must match across all devices in the cluster. The proxy addresses and port can vary.

Define external storage snapshots location

Before you can configure the external snapshot storage location, you need the following information for the machine you will use to store your data collection device (DCD) snapshots:
  • Storage-machine-IP-address
  • Storage-file-path
  • User name, password, and (optionally) the domain for the user account configured on the external storage device
  • Read/Write permissions for the storage file path

You need snapshots to perform software upgrades and to restore your old data.

When you create DCD snapshots, they need to be stored on a machine other than the DCD. You define the location for the snapshot using the BIG-IQ® Centralized Management device.

  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click the Settings button.
    The Settings screen opens to display the current state of the DCD cluster defined for this BIG-IQ device.
  4. For External Storage, click Configure.
    The External Storage popup screen opens.
  5. In the User name and Password fields, type the user name and password for the user account configured on the external storage device.
  6. For the Domain, type in the domain name for the user account configured on the external storage device.
  7. For the Storage Path, type the path to the external storage location.
    You can specify the device using the IP address or the host name. Additionally, you need to specify the path to the folder on the external storage device. For example:
    //<storage machine ip-address>/<storage-file-path>
    Note: Remember, the folder you specify must have full read, write, and execute permissions.
  8. To test the settings just specified, click Test.
    A message displays to tell you whether the test completes successfully. If it does not, correct the settings and permissions until it completes successfully.
  9. When the external storage is specified successfully, click Save.
The storage location should now be accessible to the all of the devices in the DCD cluster.

Define snapshot schedules

Before you define snapshot schedules, you must have defined the snapshot storage location.
Snapshots of the data sent to your data collection devices are an essential safeguard for your data. If the machine that stores the data fails, the data can be restored using these snapshots. These snapshots are created based on the snapshot schedules you define. F5 recommends that you schedule snapshots at least every 6 hours, and retain at least 4 snapshots.
Note: You perform this task on the BIG-IQ® Centralized Management device; not on the data collection device (DCD).
  1. At the top of the screen, click System.
  2. On the left, expand BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Devices.
    The BIG-IQ Data Collection Devices screen opens to display the currently defined data collection device cluster.
  3. Click the Settings button.
    The Settings screen opens to display the current state of the DCD cluster defined for this BIG-IQ device.
  4. To view the list of snapshot schedules for this device,in the External Storage & Snapshots area, for Snapshot Schedules click the View Schedules button.
    The BIG-IQ Data Collection Snapshot Schedules screen opens.
  5. To define a new snapshot schedule for this device, click , click Create.
    The New Logging Snapshot Schedules screen opens.
  6. For the Snapshot Name Prefix, type the string that you want to use to identify the snapshots created by this schedule.
    For example snapshot_.
  7. In Snapshots to Keep, specify the number of snapshots that you want to accumulate before they are deleted for space constraints.
    For example, if you specify 25, then the system will retain a maximum of 25 snapshots before it starts to delete older snapshots as new snapshots are created. You can save up to 100.
  8. Define how you want the snapshots scheduled.
    Option Description
    Schedule the interval at which you want to create snapshots:

    You schedule the system to take snapshots indefinitely. Snapshots are created at the frequency you specify.

    1. Select Repeat Interval.
    2. Specify the Snapshot Frequency.
    3. Select a time increment.

    For example, if you set the frequency to 6 and Hours, the first DCD snapshot is taken immediately (on Save). Subsequent snapshots are taken every 6 hours.

    Schedule specific days on which you want to create snapshots:

    You schedule the system to take snapshots on specific days.

    1. Select Days of the Week.
    2. For the Days of the Week setting, select the days on which you want backups to occur.
    3. For the Start Date, select the time (date, hour, minute, and AM or PM) on which you want backups to start.
  9. Click Save & Close to save the new schedule.

Overview of configuring the data collection device to BIG-IP device connection

The workflow to configure data to route from the BIG-IP® devices to your data collection device (DCD) cluster depends on the type of data you want to collect.

  • To collect statistics data, refer to Discover and activate a data collection device.
  • To collect Access Policy Manager® data, refer to Configuring remote logging for Access Policy Manager.
  • To collect Fraud Protection Services data, refer to Configuring BIG-IP FPS devices to route alerts to a data collection device.
  • To collect Web Application Security data, refer to:
    • Configuring the BIG-IP logging profile
    • Virtual servers that remote logging uses to route event logs
    • Assigning the logging profile to a virtual server
    .
Configure remote logging for Access Policy Manager
BIG-IP® devices that you configure for remote logging send Access reporting and SWG log report data to the BIG-IQ® data collection device for storage and management.
  1. At the top left of the screen, click Monitoring > DASHBOARDS > Access .
  2. Click Remote Logging Configuration.
    The Remote Logging Configuration screen opens to display all of the discovered BIG-IP devices that are provisioned with the Access service.
  3. Select the BIG-IP devices for which you want to enable remote logging, and then click Configure.
    The hostname of the primary data collection device is displayed, and the status changes to let you know whether the enable request was successful.
Configuring BIG-IP FPS devices to route alerts to a data collection device

The BIG-IP® device that generates Fraud Protection Service alerts must be configured to send its alerts to the data collection device (DCD). This process is documented in a separate guide. The guide F5® Fraud Protection Service: Configuration, Version 13.0 provides complete setup instructions for using FPS on a BIG-IP® system. Complete the standard setup as documented in the guide, except when you configure the alert server pool, add your DCDs to an alerts pool using their internal self IP addresses.

Note: Although DCDs use their own version of load balancing to level the data stored on each node, it is best practice to configure the BIG-IP pool members with a load balancing method that ensures smooth traffic flow to the DCDs. The load balancing method you configure should:
  • Distribute traffic between the nodes.
  • Ensure that, if a DCD goes offline, the BIG-IP device must still be able send traffic to the available DCDs without dropping alerts.

The default port to specify is 8008, but you can use a different port if your DCD is configured for it. To ensure that alerts are received even if one DCD goes down, specify at least one alternative DCD.

Configure the BIG-IP logging profile
For Web Application Security users, this is the first of three tasks required to route the BIG-IP® event logs to a BIG-IQ® data collection device. You configure the BIG-IP system by creating a logging profile and assigning the logging profile to a virtual server, and then deploying it to the BIG-IP system. The logging profile defines the content of the events, and identifies the data collection device to which the events are sent.
  1. At the top of the screen, click Configuration.
  2. On the left, click SECURITY > Shared Security > Logging Profiles .
    The Logging Profiles screen opens to display the logging profiles that have been configured on this device.
  3. On the Logging Profiles screen, click Create.
    The New Logging Profile screen opens, showing the Properties information.
  4. On the Properties screen, edit as appropriate:
    1. In the Name field, type a unique name for this new profile. This field is required.
    2. For the Description, you can specify an optional description for the logging profile.
    3. For the Partition, you can specify the partition to which the logging profile belongs. Only users with access to a partition can view the objects (such as the logging profile) that it contains. If the logging profile resides in the Common partition, all users can access it. Although this field is pre-populated with Common by default, you can set the partition when creating logging profiles by typing a unique name for the partition.
      Note: The partition with the name you specify must already exist on the BIG-IP device. No whitespace is allowed in the partition name.
    4. To specify the devices to which you want to deploy this logging profile, select the devices in the Available list, and click the right arrow to add them to the Selected list.
  5. On the left, click Application Security, and then select the Enabled check box.
    The screen displays the Application Security settings.
    1. Select the Remote Storage Enabled check box.
      The screen displays additional settings, and the Local Storage option becomes active.
    2. Clear the Local Storage check box.
    3. Specify the appropriate Logging Format.
      • If the BIG-IP device runs version 12.0 or later, select BIG-IQ.
      • If the BIG-IP device runs a version earlier than 12.0, select Comma-Separated Values. Several new settings appear.
        • For Storage Format, select User Defined.
        • In the Selected field, paste the following text:
          unit_hostname="%unit_hostname%",management_ip_address="%management_ip_address%",
          http_class_name="%http_class_name%",web_application_name="%http_class_name%",policy_name="%policy_name%",
          policy_apply_date="%policy_apply_date%",violations="%violations%",support_id="%support_id%",
          request_status="%request_status%",response_code="%response_code%",ip_client="%ip_client%",
          route_domain="%route_domain%",method="%method%",protocol="%protocol%",query_string="%query_string%",
          x_forwarded_for_header_value="%x_forwarded_for_header_value%",sig_ids="%sig_ids%",sig_names="%sig_names%",
          date_time="%date_time%",severity="%severity%",attack_type="%attack_type%",geo_location="%geo_location%",
          ip_address_intelligence="%ip_address_intelligence%",username="%username%",session_id="%session_id%",
          src_port="%src_port%",dest_port="%dest_port%",dest_ip="%dest_ip%",sub_violations="%sub_violations%",
          virus_name="%virus_name%",uri="%uri%",request="%request%",violation_details="%violation_details%",
          header="%headers%",response="%response%
          
          Note: The line breaks in the example above were necessary due to screen width; remove all of them after you paste this data. It must be a single string with no white space.
    4. For Protocol, select TCP.
    5. For the Server Addresses settings, specify the address you want to use:
      1. In the IP Address field, type the data collection node's management IP address.
      2. Specify the port to use for your data.
        • If you are setting up a logging profile for Web Application Security, type 8514 in the Port field.
        • If you are setting up a logging profile for Fraud Protection Service, type 8008 in the Port field.
      3. Click the Add button to add the address and port to the list of servers.
    6. For the Maximum Entry Length, select 64k.
    7. In the Storage Filter area, from the Request Type list, select All requests.
  6. If you want to specify Protocol Security options, on the left click Protocol Security, then select the Enabled check box: the Protocol Security settings display. Edit as appropriate.
  7. If you want to specify Network Firewall options, on the left click Network Firewall, then select the Enabled check box: the Network Firewall settings display. Edit as appropriate.
  8. If you want to specify Network Address Translation options, on the left click Network Address Translation, then select the Enabled check box: the Network Address Translation settings display. Edit as appropriate.
  9. If you want to specify DoS Protection options, on the left click DoS Protection, then select the Enabled check box: the DoS Protection settings display. Edit as appropriate.
  10. Click Save & Close to save the new profile.
The new logging profile is added to the list of profiles defined on this device.
Before you can begin using this profile, you must assign it to a virtual server and then deploy the virtual server to the BIG-IP device.
Virtual servers that remote logging uses to route alert or event logs

You can either create a new virtual server on the BIG-IP® device that creates the alert or event, or you can use a virtual server that already exists on that device.

Creating a virtual server for remote logging
If the device for which you are configuring remote logging does not have a virtual server, you need to create one.
  1. At the top of the screen, click Configuration.
  2. On the left, expand LOCAL TRAFFIC.
  3. Under LOCAL TRAFFIC, select Virtual Servers.
    The screen displays a list of virtual servers defined on this device.
  4. Click Create.
    The Virtual Servers - New Item screen opens.
  5. In the Name field, type in a name for the virtual server you are creating.
  6. From the Device list, select the device on which to create the virtual server.
  7. In the Description field, type in a brief description for the virtual server you are creating.
  8. For the Destination Address, type the IP address of the destination you want to add to the Destination list.

    The format for an IPv4 address is I<a>.I<b>.I<c>.I<d>. For example, 172.16.254.1.

    The format for an IPv6 address is I<a>:I<b>:I<c>:I<d>:I<e>:I<f>:I<g>:I<h>..

    For example, 2001:db8:85a3:8d3:1319:8a2e:370:7348.
  9. In the Service Port field, type a service port number, or select a type from the list.
    When you select a type from the list, the value in the Service Port field changes to reflect the associated default, which you can change.
  10. Click Save.
    The system creates the new virtual server with the settings you specified.
  11. Click Save to save the assignment. Or, click Save & Close to save the assignment and return to the Virtual Servers screen.
A virtual server that can be used to route alert or event data to the logging node is created for the BIG-IP® device.
Before the BIG-IP device can actually use this new virtual server, you must deploy it to the device.
Assign the logging profile to a virtual server
After configuring a logging profile on the BIG-IQ® system, you must assign it to a virtual server and deploy it to the BIG-IP® device from which you want to collect event logs.
  1. At the top of the screen, click Configuration.
  2. On the left, click SECURITY > Shared Security > Virtual Servers .
    The screen displays a list of virtual servers that are configured with devices that have been provisioned and discovered.
  3. On the Virtual Servers screen, click the name of the virtual server you want to use.
    The Virtual Servers - Properties screen opens.
  4. From the Log Profiles list, under Available, click a logging profile and move it to the Selected list.
  5. Click Save & Close to save the assignment and return to the Virtual Servers screen.
The virtual server is now associated with the logging profile.
Before the BIG-IP system(s) can start sending alert or event logs to the data collection device, you must deploy the changes you just made to the BIG-IP device.