Applies To:

Show Versions Show Versions

Manual Chapter: Setting Up Application Statistics Collection
Manual Chapter
Table of Contents   |   Next Chapter >>

What is Analytics?

Analytics (also called Application Visibility and Reporting or AVR) is a module on the BIG-IP® system that you can use to visually analyze the performance of web applications, TCP traffic, DNS traffic, FastL4, and overall system statistics. The statistics are displayed in graphical charts where you can drill down for more specific details to better understand network performance on certain devices, IP addresses, memory and CPU utilization, and so on. You can focus the statistics in the charts on different categories such as applications or virtual servers.

For HTTP traffic, Analytics provides detailed metrics such as transactions per second, server and client latency, request and response throughput, and sessions. You can view metrics for applications, virtual servers, pool members, transaction outcomes,URLs, specific countries, and additional detailed statistics about application traffic running through the BIG-IP system.

Transaction counters for response codes, user agents, HTTP methods, countries, and IP addresses provide statistical analysis of HTTP traffic that is going through the system. You can capture HTTP traffic for examination, and have the system send alerts so you can troubleshoot problems and immediately react to sudden changes.

For TCP and FastL4 traffic, reports show details about RTT (round trip time), goodput, connections, and packets. For TCP, you can also view statistics for delay analysis. Within these categories, you can display information by the requests side, applications, virtual servers, remote host IP addresses, subnet addresses, next hops, countries, cities, continents, or user provided keys (from the TCP::analytics iRule). You can use the reports to gather information about TCP flows to better understand what is happening on your network. For example, you could view the charts by applications, then examine RTT averages, packet loss, and connection length to investigate user complaints about a slowdown.

You specify the type of traffic to monitor using different Analytics profiles. To view web application statistics, you use an HTTP Analytics profile, and to view TCP or FastL4 statistics, you use a TCP Analytics profile. Viewing system statistics does not require an Analytics profile.

Using remote logging capabilities with Analytics, your company can consolidate statistics gathered from multiple BIG-IP appliances onto syslog servers or SIEM devices, such as Splunk. A report scheduler allows you to periodically send email to users with specific types of reports that you design.

About HTTP Analytics profiles

An HTTP Analytics profile is a set of definitions that determines the circumstances under which the system gathers, logs, notifies, and graphically displays information regarding traffic to an application. You select an HTTP Analytics profile for each application you want to monitor. You associate the HTTP Analytics profile with one or more virtual servers used by the application. Each virtual server can have one HTTP and/or one TCP Analytics profile associated with it.

In the HTTP Analytics profile, you customize:

  • What statistics to collect
  • Where to collect data (locally, remotely, or both)
  • Whether to capture the traffic itself
  • Whether to send notifications

The BIG-IP® system includes a default HTTP Analytics profile called analytics. It serves as the parent of all other HTTP Analytics profiles that you create on the system. You can modify the default profile, or create custom HTTP Analytics profiles for each application if you want to track different data for each one. Certain settings, such as SMTP Configuration, Transaction Sampling, and the Subnets list, can only be set in the default HTTP Analytics profile.

Statistics > Analytics displays the HTTP Overview by default, which shows:

  • Average transactions per second
  • Average request throughput
  • Average response throughput
  • Average server latency
  • Average page load time
  • Average concurrent sessions
  • Average new sessions

Charts shown on the HTTP Overview screen include the application data saved for all HTTP Analytics profiles associated with iApps application services and virtual servers on the system. You can filter the HTTP information by many different criteria, such as by application or URL. You can also drill down into the specifics on the charts, and use the options to further refine the information in the charts.

Overview: Collecting application statistics

This implementation describes how to set up the BIG-IP® system to collect application performance statistics. The system can collect application statistics locally, remotely, or both. You use these statistics for troubleshooting and improving application performance.

You can collect application statistics for one or more virtual servers or for an iApps® application service. If virtual servers are already configured, you can specify them when setting up statistics collection. If you want to collect statistics for an iApps application service, you should first set up statistics collection, creating an HTTP Analytics profile, and then create the application service.

The system can send alerts regarding the statistics when thresholds are exceeded, and when they cross back into the normal range. You can customize the threshold values for transactions per second, latency, page load time, and throughput.

Task Summary

Customizing the default HTTP Analytics profile

The Application Visibility and Reporting (AVR) module includes a default HTTP Analytics profile called analytics. You can edit the settings in the default profile so it uses the values you want.

Certain information can be specified only in the default HTTP Analytics profile: the SMTP configuration (a link to an SMTP server), transaction sampling (whether enabled or not), and subnets (assigning names to be used in the reports). To edit these values, you need to open and edit the default profile.

  1. On the Main tab, click Local Traffic > Profiles > Analytics > HTTP Analytics .
    Tip: To have the Analytics listed, you need to provision Application Visibility and Reporting (AVR) first.
    The Profiles: Analytics screen opens.
  2. Click the profile called analytics.
    The configuration screen for the default HTTP Analytics profile opens.
  3. For the Statistics Logging Type setting, verify that Internal is selected. If it is not, select it.
    Selecting Internal causes the system to store statistics locally, and you can view the charts on the system by starting at the Main tab, and clicking Statistics > Analytics .
  4. To send email alerts, specify an SMTP Configuration.
    You can change the SMTP configuration only in the default profile. It is used globally for the system. If no configuration is available, click Create to create one.
  5. For the Notification Type setting, select how you want the system to send alerts and notifications.
    Option Description
    Syslog Select Syslog if you want the system to send notification and alert messages to the local log system. You can view the messages on the System > Logs > Local Traffic screen.
    SNMP Select SNMP if you want the system to send notification and alert messages as SNMP traps. You can create the trap by clicking Configuration can be found here ( System > SNMP > Traps > Destination ). Enabling SNMP automatically sets up Syslog notifications, too.
    E-mail Select E-mail if you want the system to send notification and alert messages to email addresses. Type each email address in the Notification E-Mails field, and click Add to create the list. This option requires that the default analytics profile includes an SMTP configuration.
    When you select a notification type, the screen displays the Alerts and Notifications Configuration area, where you can indicate the criteria for alerts and notifications.
  6. If you want the system to perform traffic sampling, make sure that for Transaction Sampling, the Sample check box is selected.
    You can change this setting only in the default profile.
    Tip: Sampling improves system performance. F5 recommends that you enable sampling if you generally use more than 50 percent of the system CPU resources, or if you have at least 100 transactions in 5 minutes for each entity.
  7. If you want the system to collect and display statistics, according to the expressions written in an iRule, select the Publish iRule Statistics check box.
    The iRule statistics can be viewed per Analytics profile on the command line by typing ISTATS dump.
    Important: For the system to collect iRule statistics, you must also write an iRule describing which statistics the system should collect.
  8. In the Associated Virtual Servers area, specify the virtual servers for which to capture application statistics:
    1. For the Virtual Servers setting, click Add.
    2. From the Select Virtual Server popup that displays, select the virtual servers to include and then click Done.
    Note: Only virtual servers previously configured with an HTTP profile display in the list (because the data being collected applies to HTTP traffic). Also, you can assign only one HTTP Analytics profile to a virtual server; therefore, the list displays only virtual servers that have not been assigned an Analytics profile.
    Special considerations apply if using Analytics on a BIG-IP system with both Application Security Manager™ and Access Policy Manager®, where security settings (in Portal Access webtop or an iRule) redirect traffic from one virtual server to another. In this case, you need to attach the HTTP Analytics profile to the second virtual server to ensure that the charts show accurate statistics.
  9. In the Statistics Gathering Configuration area, for Collected Metrics, select additional statistics you want the system to collect from the requests:
    Option Description
    Max TPS and Throughput Collects statistics showing the maximum number of transactions occurring per second and the amount of traffic moving through the system (maximum request and response throughput is collected and recorded separately).
    Page Load Time Tracks how long it takes an application user to get a complete response from the application, including network latency and completed page processing.
    Note: End-user response times and latencies can vary significantly based on geography and connection types.
    User Sessions Stores the number of unique user sessions. For Timeout, select the number of minutes of user inactivity to allow before the system considers the session to be over.

    For Cookie Secure Attribute, specify whether to secure session cookies. Options are Always, the secure attribute is always added to the session cookie; Never, the secure attribute is never added to the session cookie; or Only SSL, the secure attribute is added to the session cookie only when the virtual server has a client SSL profile (the default value).

    By default, the system collects many metrics, including TPS, throughput, server latency, response time, network latency, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  10. In the Statistics Gathering Configuration area, for Collected Entities, select additional entities to collect statistics for each request.
    Option Description
    URLs Collects the requested URLs.
    Countries Saves the name of the country where the request came from, and is based on the client IP address criteria.
    Client IP Addresses Saves the IP address where the request originated. The address saved also depends on whether the request has an XFF (X-forwarded-for) header and whether the HTTP profile accepts XFF headers.
    Client Subnets Saves statistics for predefined client subnets. Client subnets can be added in the Subnets area of the default HTTP Analytics profile.
    Response Codes Saves HTTP response codes that the server returned to requesters.
    User Agents Saves information about browsers making the request.
    Methods Saves HTTP methods in requests.
    By default, the system collects many entity statistics, including virtual servers, pool members, browser names, operating system, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  11. If you are collecting statistics for Client Subnets, you can name the subnets so the reports show a name (such as a department name) instead of an IP address. To do this, add the subnets:
    1. For the Add New Subnet setting Name field, type the name to use, and in the Mask field, type the IP address of the subnet.
    2. Click Add.
    The subnets are added to the list of Active Subnets. If displaying relevant data, the names of the subnets appear in the Analytics statistics.
  12. Click Update to save your changes.
Statistics are collected for traffic going to the virtual servers specified in this profile.
You can create new HTTP Analytics profiles if you need to. New Analytics profiles inherit their values from the default Analytics profile. You can modify the values in the new profiles (except the ones that are set only in the default profile such as SMTP configuration). For example, you might want to send reports about two different applications to different managers. So you could have different emails listed in the notification type settings in two Analytics profiles.

Collecting application statistics locally

You need to provision the Application Visibility and Reporting (AVR) module before you can collect application statistics locally.
You can configure the BIG-IP® system to collect specific application statistics locally.
  1. On the Main tab, click Local Traffic > Profiles > Analytics > HTTP Analytics .
    Tip: If Analytics is not listed, this indicates that Application Visibility and Reporting (AVR) is not provisioned, or you do not have rights to create profiles.
    The HTTP Analytics screen opens.
  2. Click Create.
    The New HTTP Analytics profile screen opens.
  3. In the Profile Name field, type a unique name for the Analytics profile.
  4. From the Parent Profile list, select the profile from which you want to inherit settings.
    The default profile is often used as the parent profile.
    The new profile inherits the values from the parent profile. If the parent is changed, the inherited values in the new profile also change.
  5. Select the Custom check box.
  6. For the Statistics Logging Type setting, verify that Internal is selected. If it is not, select it.
    Selecting Internal causes the system to store statistics locally, and you can view the charts on the system by starting at the Main tab, and clicking Statistics > Analytics .
  7. You can use the default values for the rest of the General Configuration settings.
  8. In the Associated Virtual Servers area, specify the virtual servers for which to capture application statistics:
    1. For the Virtual Servers setting, click Add.
    2. From the Select Virtual Server popup that displays, select the virtual servers to include and then click Done.
    Note: Only virtual servers previously configured with an HTTP profile display in the list (because the data being collected applies to HTTP traffic). Also, you can assign only one HTTP Analytics profile to a virtual server; therefore, the list displays only virtual servers that have not been assigned an Analytics profile.
    Special considerations apply if using Analytics on a BIG-IP system with both Application Security Manager™ and Access Policy Manager®, where security settings (in Portal Access webtop or an iRule) redirect traffic from one virtual server to another. In this case, you need to attach the HTTP Analytics profile to the second virtual server to ensure that the charts show accurate statistics.
  9. In the Statistics Gathering Configuration area, select the Custom check box.
  10. In the Statistics Gathering Configuration area, for Collected Metrics, select additional statistics you want the system to collect from the requests:
    Option Description
    Max TPS and Throughput Collects statistics showing the maximum number of transactions occurring per second and the amount of traffic moving through the system (maximum request and response throughput is collected and recorded separately).
    Page Load Time Tracks how long it takes an application user to get a complete response from the application, including network latency and completed page processing.
    Note: End-user response times and latencies can vary significantly based on geography and connection types.
    User Sessions Stores the number of unique user sessions. For Timeout, select the number of minutes of user inactivity to allow before the system considers the session to be over.

    For Cookie Secure Attribute, specify whether to secure session cookies. Options are Always, the secure attribute is always added to the session cookie; Never, the secure attribute is never added to the session cookie; or Only SSL, the secure attribute is added to the session cookie only when the virtual server has a client SSL profile (the default value).

    By default, the system collects many metrics, including TPS, throughput, server latency, response time, network latency, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  11. In the Statistics Gathering Configuration area, for Collected Entities, select additional entities to collect statistics for each request.
    Option Description
    URLs Collects the requested URLs.
    Countries Saves the name of the country where the request came from, and is based on the client IP address criteria.
    Client IP Addresses Saves the IP address where the request originated. The address saved also depends on whether the request has an XFF (X-forwarded-for) header and whether the HTTP profile accepts XFF headers.
    Client Subnets Saves statistics for predefined client subnets. Client subnets can be added in the Subnets area of the default HTTP Analytics profile.
    Response Codes Saves HTTP response codes that the server returned to requesters.
    User Agents Saves information about browsers making the request.
    Methods Saves HTTP methods in requests.
    By default, the system collects many entity statistics, including virtual servers, pool members, browser names, operating system, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  12. Click Finished.
The BIG-IP system collects the statistics specified in the Analytics profile. You can view the statistics by clicking Statistics > Analytics .

Collecting application statistics remotely

You need to provision the Application Visibility and Reporting (AVR) module before you can collect application statistics remotely. To specify where the BIG-IP® system sends log messages remotely, you must have set up logging and created a publisher.
You can configure the BIG-IP system to collect application statistics and store them remotely on Syslog servers or SIEM devices, such as Splunk.
  1. On the Main tab, click Local Traffic > Profiles > Analytics > HTTP Analytics .
    Tip: If Analytics is not listed, this indicates that Application Visibility and Reporting (AVR) is not provisioned, or you do not have rights to create profiles.
    The HTTP Analytics screen opens.
  2. Click Create.
    The New HTTP Analytics profile screen opens.
  3. In the Profile Name field, type a unique name for the Analytics profile.
  4. Select the Custom check box.
  5. For the Statistics Logging Type setting, select the External check box.
    Unless you want to view statistics locally, too, you can clear the Internal check box.
    The Remote Publisher setting displays, below the Traffic Capturing Logging Type setting.
  6. If you want the system to capture traffic, for the Traffic Capturing Logging Type setting, specify whether to store the traffic locally or on a remote server.
    Option Description
    Internal Specifies that the system captures a portion of traffic and stores it locally. You can view the captured data on the System > Logs > Captured Transactions screen.
    External Specifies that the system captures a portion of traffic and stores it on a remote server.
    When you select the traffic capturing logging type, the screen displays the Capture Filter area, where you can indicate exactly what information to sample and log.
  7. From the Remote Publisher list, select the publisher that includes the destination to which you want to send log messages.
    Tip: Refer to External Monitoring of BIG-IP® Systems: Implementations for details.
  8. If you want the system to send email notifications, review the SMTP Configuration field to ensure that a configuration is specified and not the value None.
    You can configure SMTP only in the default Analytics profile. If it is not configured, you can save the profile and edit the default profile where you can select an existing SMTP configuration or create a new one. (If you click the analytics link without saving the new profile you are working on, you will lose the unsaved changes.)
  9. For the Notification Type setting, select how you want the system to send alerts and notifications.
    Option Description
    Syslog Select Syslog if you want the system to send notification and alert messages to the local log system. You can view the messages on the System > Logs > Local Traffic screen.
    SNMP Select SNMP if you want the system to send notification and alert messages as SNMP traps. You can create the trap by clicking Configuration can be found here ( System > SNMP > Traps > Destination ). Enabling SNMP automatically sets up Syslog notifications, too.
    E-mail Select E-mail if you want the system to send notification and alert messages to email addresses. Type each email address in the Notification E-Mails field, and click Add to create the list. This option requires that the default analytics profile includes an SMTP configuration.
    When you select a notification type, the screen displays the Alerts and Notifications Configuration area, where you can indicate the criteria for alerts and notifications.
  10. In the Associated Virtual Servers area, specify the virtual servers for which to capture application statistics:
    1. For the Virtual Servers setting, click Add.
    2. From the Select Virtual Server popup that displays, select the virtual servers to include and then click Done.
    Note: Only virtual servers previously configured with an HTTP profile display in the list (because the data being collected applies to HTTP traffic). Also, you can assign only one HTTP Analytics profile to a virtual server; therefore, the list displays only virtual servers that have not been assigned an Analytics profile.
    Special considerations apply if using Analytics on a BIG-IP system with both Application Security Manager™ and Access Policy Manager®, where security settings (in Portal Access webtop or an iRule) redirect traffic from one virtual server to another. In this case, you need to attach the HTTP Analytics profile to the second virtual server to ensure that the charts show accurate statistics.
  11. In the Statistics Gathering Configuration area, for Collected Metrics, select additional statistics you want the system to collect from the requests:
    Option Description
    Max TPS and Throughput Collects statistics showing the maximum number of transactions occurring per second and the amount of traffic moving through the system (maximum request and response throughput is collected and recorded separately).
    Page Load Time Tracks how long it takes an application user to get a complete response from the application, including network latency and completed page processing.
    Note: End-user response times and latencies can vary significantly based on geography and connection types.
    User Sessions Stores the number of unique user sessions. For Timeout, select the number of minutes of user inactivity to allow before the system considers the session to be over.

    For Cookie Secure Attribute, specify whether to secure session cookies. Options are Always, the secure attribute is always added to the session cookie; Never, the secure attribute is never added to the session cookie; or Only SSL, the secure attribute is added to the session cookie only when the virtual server has a client SSL profile (the default value).

    By default, the system collects many metrics, including TPS, throughput, server latency, response time, network latency, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  12. In the Statistics Gathering Configuration area, for Collected Entities, select additional entities to collect statistics for each request.
    Option Description
    URLs Collects the requested URLs.
    Countries Saves the name of the country where the request came from, and is based on the client IP address criteria.
    Client IP Addresses Saves the IP address where the request originated. The address saved also depends on whether the request has an XFF (X-forwarded-for) header and whether the HTTP profile accepts XFF headers.
    Client Subnets Saves statistics for predefined client subnets. Client subnets can be added in the Subnets area of the default HTTP Analytics profile.
    Response Codes Saves HTTP response codes that the server returned to requesters.
    User Agents Saves information about browsers making the request.
    Methods Saves HTTP methods in requests.
    By default, the system collects many entity statistics, including virtual servers, pool members, browser names, operating system, and so on. You can select the ones here in addition to the ones already collected once the Analytics profile is attached to one or more virtual servers.
  13. If one of the Traffic Capturing Logging Type check boxes is selected, in the Capture Filter area, adjust the settings to specify criteria to determine what application traffic to capture.
    Tip: You can use the captured information for troubleshooting purposes.
  14. Click Finished.
The BIG-IP system collects statistics regarding application traffic described by the Analytics profile and stores the statistics on a separate remote management system, where you can view the information.

Getting application performance alerts

Before you can configure the system to send alerts concerning statistics, you need to have created an Analytics profile to collect application statistics locally (Statistics Logging Type must have Internal selected). To set up email alerts, the default analytics profile must specify an SMTP configuration.
You can configure the BIG-IP® system to send alerts concerning local application statistics based on threshold values that you set. The system sends notifications when threshold values are breached, and when they return to normal. Therefore, it is a good idea to get familiar with the typical statistics for the web application before attempting to set up alerts and notifications. When you understand the typical values, you can configure the system to alert you of limiting system situations, such as system overload.
Note: End user response times and latencies can vary significantly based on geography and connection types, which makes it difficult to set an accurate alerting threshold for page load times.
  1. On the Main tab, click Local Traffic > Profiles > Analytics > HTTP Analytics .
    Tip: To have the Analytics listed, you need to provision Application Visibility and Reporting (AVR) first.
    The Profiles: Analytics screen opens.
  2. Click the name of a previously created Analytics profile, or create a new one.
  3. Select the Custom check box.
  4. For the Statistics Logging Type setting, verify that Internal is selected. If it is not, select it.
    Selecting Internal causes the system to store statistics locally, and you can view the charts on the system by starting at the Main tab, and clicking Statistics > Analytics .
  5. To send email alerts, specify an SMTP Configuration (this can only be done on the default analytics profile).
    If you created a new profile, configure SMTP later.
  6. For the Notification Type setting, select how you want the system to send alerts and notifications.
    Option Description
    Syslog Select Syslog if you want the system to send notification and alert messages to the local log system. You can view the messages on the System > Logs > Local Traffic screen.
    SNMP Select SNMP if you want the system to send notification and alert messages as SNMP traps. You can create the trap by clicking Configuration can be found here ( System > SNMP > Traps > Destination ). Enabling SNMP automatically sets up Syslog notifications, too.
    E-mail Select E-mail if you want the system to send notification and alert messages to email addresses. Type each email address in the Notification E-Mails field, and click Add to create the list. This option requires that the default analytics profile includes an SMTP configuration.
    When you select a notification type, the screen displays the Alerts and Notifications Configuration area, where you can indicate the criteria for alerts and notifications.
  7. In the Alerts and Notifications Configuration area, for the Add New Rule setting, define the rules that determine when the system sends alerts. Note that you cannot add overlapping rules, for example, two rules that request an alert when average TPS is greater than 100 and greater than 50 for 200 seconds.
    1. For Alert when, select the condition under which you want to send an alert.
    2. Select below or above, type an integer that represents the threshold value, and type the number of seconds (an integer, 300 or greater,) during which the rule has to apply.
    3. Select the granularity level to which the threshold applies: traffic sent to an Application, a Virtual Server, or a Pool Member.
    4. Click Add.
      The rule is added to the list of Active Rules.
    Continue to add as many rules as you want to specify conditions under which you want to be alerted.
  8. Click Update.
  9. If SNMP is not configured on the BIG-IP system and you want to send SNMP traps, configure it now:
    1. In the General Configuration area, for the Notification Type setting, next to SNMP, click the link.
      The SNMP Traps Destination screen opens.
    2. Click Create.
    3. Configure the version, community name, destination IP address, and port.
    4. Click Finished.
  10. If you need to configure SMTP (if sending alerts by email), click the default analytics profile on the Profiles: Analytics screen.
    1. For SMTP Configuration, select an existing configuration.
    2. If no SMTP configurations are listed, click the here link to create one. When you are done, you need to select the configuration you created in the default analytics profile.
Based on the rules you configured and the notification type, the system sends an alert when thresholds are breached and when they cross back from the threshold.

Creating an SMTP server configuration

You specify the SMTP server configuration so that you can send emails through an SMTP server.
  1. On the Main tab, click System > Configuration > Device > SMTP .
  2. Click the Create button.
    The New SMTP Configuration screen opens.
  3. In the Name field, type a name for the SMTP server that you are creating.
  4. In the SMTP Server Host Name field, type the fully qualified domain name for the SMTP server host.
  5. In the SMTP Server Port Number field, type a port number.
    For no encryption or TLS encryption, the default is 25. For SSL encryption, the default is 465.
  6. In the Local Host Name field, type the host name used in the SMTP headers in the form of a fully qualified domain name.
    This host name is not the same as the BIG-IP® system's host name.
  7. In the From Address field, type the email address that you want displayed as the reply-to address for the email.
  8. From the Encrypted Connection list, select the encryption level required for the SMTP server.
  9. To require that the SMTP server validates users before allowing them to send email, select the Use Authentication check box, and type the user name and password required to validate the user.
  10. Click the Finish button.
You can now configure the system to use this SMTP server to send emails. For the SMTP mailer to work, you must make sure the SMTP server is on the DNS lookup server list, and configure the DNS server on the BIG-IP® system.
Table of Contents   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)