Applies To:

Show Versions Show Versions

Manual Chapter: Prerequisites
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Prerequisites

What should I know about disk space before estimating resource requirements?

In addition to the alert rate, consider the number of devices you plan to manage, the data retention policy you plan to use and the average size of the alerts you plan to store on the DCD. When you configure the retention and rotation settings for your alert data, you must consider the available storage on the DCD. Although DCD VE devices allow for expansion of the alert partition, be aware that if you exceed 50% of the available disk space (which is not the same as raw disk space), you will not be able to upgrade your BIG-IQ DCD cluster. After deployment, a certain amount of the disk space is consumed by the BIG-IQ system software and is unavailable.

Determine available disk space

If you are planning to upgrade a deployed DCD, you can use the following steps to determine the amount of available disk space:
  1. Use SSH to log in to your DCD as root.
  2. Check the disk space available by running the vgdisplay command:
    For example:
    
                        vgdisplay
                         --- Volume group ---
                        VG Name               vg-db-sda
                        System ID
                        Format                lvm2
                        Metadata Areas        1
                        Metadata Sequence No  179
                        VG Access             read/write
                        VG Status             resizable
                        MAX LV                0
                        Cur LV                16
                        Open LV               7
                        Max PV                0
                        Cur PV                1
                        Act PV                1
                        VG Size               498.50 GiB
                        PE Size               4.00 MiB
                        Total PE              127617
                        Alloc PE / Size       27936 / 109.12 GiB
                        Free  PE / Size       99681 / 389.38 GiB
                        
    

    In this example, the total disk space (VG Size) is 498.50 GB

  3. Identify how the disk space is allocated across your partitions by running the lvs command:
    For example:
    
                        [root@vBIGIQ-LN1:Active:Standalone] config # lvs
                          LV            VG        Attr      LSize   …
                          dat.log.1     vg-db-vda -wi-ao---   7.00g
                            dat.maint.1   vg-db-vda -wi-a---- 300.00m
                            dat.share.1   vg-db-vda -wi-ao---  28.00g
                            dat.swapvol.1 vg-db-vda -wi-ao---   1.00g
                            set.1._config vg-db-vda -wi-a----   3.17g
                            set.1._usr    vg-db-vda -wi-a----   3.29g
                            set.1._var    vg-db-vda -wi-a----  10.00g
                            set.1.root    vg-db-vda -wi-a---- 440.00m
                            set.2._config vg-db-vda -wi-ao---   3.17g
                            set.2._usr    vg-db-vda -wi-ao---   3.29g
                            set.2._var    vg-db-vda -wi-ao---  10.00g
                            set.2.root    vg-db-vda -wi-ao--- 440.00m
    
                        
    
    There are four entries to note in this response:
    • dat.log.1 (highlighted in yellow) displays the size of the /var/log partition that is used to store the collected log data.
    • dat.share.1 (highlighted in aqua) displays the size of the /shared partition that is used for ISOs and backups on the BIG-IQ system.
    • set.1._var (highlighted in green) displays the size of the /var partition for HD1.1 that is used for the initial software installation.
    • set.2._var (also highlighted in green) displays the size of the /var partition for HD1.2 that is used for the upgrade installation. Note that these two partitions on the active and upgrade volumes must be the same size (as in this example).
  4. If you determine you need additional disk space, increase the disk space (for both volumes) to the size you need.
    Refer to Resizing Disk Space on BIG-IQ Virtual Edition on support.f5.com for details.

What resources do I need if I’m just collecting statistics?

We’ve created a spreadsheet and a script to help you determine how many DCDs you need (and how much storage you need on each one) to handle the data generated by the BIG-IP devices you manage. You can download these tools from downloads.f5.com.

The spreadsheet uses metrics that you provide to calculate your resource requirements.

Using the sizing tools to estimate resources for statistics collection

The sizing tools uses metrics and a script that help you to calculate your resource requirements. To use the spreadsheet, you must first download it.
  1. Log in to downloads.f5.com.
  2. Navigate to BIG-IQ Centralized Management, v5.4.0, and click I Accept to accept the EULA.
  3. Click F5_Networks_BIG_IQ_DCD_Sizing_Tools.zip.
    After a brief pause, the Download Locations screen opens.
  4. Click a download location to start the download.
  5. When the download completes, unzip the file.
    Two files will extract:
    • F5_Networks_BIG_IQ_DCD_Sizing_Tool.xlsx
    • F5_Networks_BIG_IQ_Apm_Dcd_Sizing_Metrics.sh
  6. Open the spreadsheet to the Statistics DCD Sizing Tool tab, and type values that describe the environment, policies, and modules for which you want to collect statistics.
  7. After you fill in all of the fields, the recommended number and configuration of DCDs is calculated and displayed under Sizing Recommendation for DCD Nodes. You can see the impact of your storage decisions by noting how the values under Time Layer Calculations change when you enter different values.
    The calculations performed for these elements are based on some assumptions. Bear these assumptions in mind as you complete the spreadsheet.
    • The amount of device data increases marginally as you increase the number of CPUs and interfaces on your BIG-IP devices.
    • The amount of pool and pool member data is based on an assumed average number of pool members per pool. For pools that have a larger than average number of pools, the amount of data increases as well.
    • The spreadsheet assumes you have an average of 5 events per iRule. If your average number of events per iRule is lower, storage requirements go down; if your average is greater, storage requirements will increase.

What resources do I need to collect ASM data?

Use this table to estimate the number of DCDs necessary to collect and display ASM event data from BIG-IP devices.

  • If you are using an off-box logging solution to log ASM events, review a week’s worth of alert logs to identify the logging load you need to support. Take note of both the average and maximum alerts per second.
  • If you are logging ASM events locally on the BIG-IP device, you can also use those logs to identify your peak event logging rates. Compare the peak rates against the alerts per second (Alerts/Sec column) to estimate the number of DCDs you need.
  • If you have not yet deployed an ASM solution in your environment, but you have an estimate of your application’s peak HTTP/HTTPS traffic requirements, you can estimate your logging needs by projecting that 10% of your peak traffic could generate logging events. You can then use that value to estimate the number of DCDs you need.
Alerts/Sec Data Redundancy Recommended DCD count
10000 No 1
14000 Yes 2
17000 Yes 3
20000 Yes 4
23000 Yes 5
26000 Yes 6
Note:
  • The sizing recommendations in this table assume that you provision your DCDs with 8 cores/CPUs and 32 GB of memory.
  • When you choose an alert rate, consider both the average and peak loads that you anticipate. If your peak loads are frequent, consider provisioning additional data-collection nodes for the increased load. Also, when you determine your peak rates, plan for growth by considering the rate at which you expect your traffic load to increase.

What resources do I need to collect APM data?

For APM®, we provide a spreadsheet and a script to help you determine the number of DCDs required to handle the data generated by the BIG-IP® devices you manage. You can download these tools from downloads.f5.com. The APM data script collects metrics for the data your APM devices generate and stores the data in a file on your BIG-IP device.

The script reads the last 10 days of logging data and provides the metrics you need for the spreadsheet. The spreadsheet uses those metrics to calculate your resource requirements. You enter information about the amount of data you expect to generate each day, and the spreadsheet calculates the number of DCDs required to handle that data.
Note: You can use the script only if you have enabled local db-logging. For information about enabling local-db logging, refer to Overview: Configuring remote high-speed APM and SWG event logging on support.f5.com.

If you use another logging mechanism, you can run reports (such as log reports or sessions reports) to get this data.

Note: Unless there has been a recent HA failure event (within the last 3 days), run this script on the active device. In this case, the active device may not have enough data and it is better to run the script on the standby device.

Even with the script, it might be tedious to collect data from each of your BIG-IP devices, so we recommend that you take the values from one of the BIG-IP devices with the most users and access profiles to account for the worst-case scenario.

Using the script to estimate resources for APM data collection

The sizing tools uses metrics and a script that help you to calculate your resource requirements. To use the spreadsheet, you must first download it.
  1. Log in to downloads.f5.com.
  2. Navigate to BIG-IQ Centralized Management, v5.4.0, and click I Accept to accept the EULA.
  3. Click F5_Networks_BIG_IQ_DCD_Sizing_Tools.zip.
    After a brief pause, the Download Locations screen opens.
  4. Click a download location to start the download.
  5. When the download completes, unzip the file.
    Two files will extract:
    • F5_Networks_BIG_IQ_DCD_Sizing_Tool.xlsx
    • F5_Networks_BIG_IQ_Apm_Dcd_Sizing_Metrics.sh
  6. After the script has run, navigate to the /var/tmp folder on the BIG-IP device and open the output file to see the collected metrics.
    
                            NUMBER_OF_EVENTLOGS_PER_DAY
                            400,000
                            -----------------------------
                            NUMBER_OF_SESSIONS_PER_DAY
                            20,000
                            -----------------------------
                            LOGS_PER_SESSION
                            20
                            -----------------------------
                            NUMBER_OF_ACCESSS_PROFILES
                            10
                            -----------------------------
                            NUMBER_OF_ACLs
                            50
                        
    
  7. Open the spreadsheet to the Access DCD Sizing Tool tab, and type the values from the script output file into the appropriate fields on the spreadsheet.
    For example, for the sample output above, you would type 20,000 in the Number of sessions per day field.
  8. After you fill in all of the fields, the recommended number and configuration of DCDs is calculated and displayed in the Recommended number of DCD nodes field.

What resources do I need to collect FPS data?

Use this table to estimate the number of DCDs necessary to collect and display FPS alert data from BIG-IP devices.

  • If you are using an off-box logging solution to log FPS events, review a week’s worth of alert logs to identify the logging load you need to support. Take note of both the average and maximum alerts per second.

To determine the peak alerts/second rate to use in the table, first estimate the peak user load for the protected application over your highest period of production hours, and then plug that value into the following calculation:

<peak user load> users/(peak duration (h)*60m*60s)≈ number of logins/sec ≈ number of alerts/sec

For example, if you estimate that there are, at most, 1 million users logging in to access your FPS environment over a 4-hour timespan, the calculation results in a login rate of:

1,000,000 users/(4h*60m*60s) = ≈ 70 logins/sec ≈ 70 alerts/sec

Although each user login does not generate an alert, even single user logins or application transactions can generate multiple alerts depending on the protection policy’s configuration. For determining the number of DCDs needed to operate during peak conditions, 70 alerts per second is a reasonable assumption for this login rate.

Alerts/Sec Data Redundancy Recommended DCD count
100 No 1
200 Yes 2
300 Yes 3
400 Yes 4
500 Yes 5
600 Yes 6
Note:
  • The sizing recommendations in this table assume that you provision your DCDs with 8 cores/CPUs and 32 GB of memory.
  • When you choose an alert rate, consider both the average and peak loads that you anticipate. If your peak loads are frequent, consider provisioning additional data-collection nodes for the increased load. Also, when you determine your peak rates, plan for growth by considering the rate at which you expect your traffic load to increase.
  • The outgoing data rate from your BIG-IP devices also depends on the FPS policy configuration, the number of simultaneous application users, and their expected transaction rate.
  • The scale numbers provided assume the use of three forwarding rules (100% syslog forwarding, 10% SOC forwarding, 10% customer forwarding), and 17000 transform rules.
Note: Increasing the number of forwarding targets or the distance between your data collection devices increases the performance requirements for each node in the DCD cluster.

What resources do I need to collect data for multiple modules?

Use this process to estimate the number of DCDs that you need to collect and display more than one kind of data from your BIG-IP devices.

  1. Perform the estimate for each module independently.
  2. Record the maximum number of DCDs recommended by any one estimate.
  3. Sum the total storage needed by all estimates.
  4. Divide the result in step 3 by the result of step 2.

For example:

You’ve performed the estimate for an ASM module, and the result is 2 DCDs and 60GB of storage.

You’ve performed the estimate for Statistics, and the result is 3 DCDs and 120GB of storage.

In this example, the maximum number of DCDs is 3 and the sum of the total storage is 180GB. Consequently, the recommendation is 3 DCDs (the maximum, from the statistics estimate) and 60GB storage (The sum of both, divided by the recommended number of DCDs).

Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.

Additional Comments (optional)