Applies To:

Show Versions Show Versions

Manual Chapter: Reporting Usage Data to an External Analytics Server
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Overview: Reporting usage data to an external analytics server

In Policy Enforcement Manager, you can create a rule within an enforcement policy that instructs the system to send usage data in high-speed logging (HSL) format to an external analytics server. The rule specifies what type of reporting data you are interested in, and one of the actions it can take with the traffic is to send the information collected about it for processing to a centralized analytics server.

The system sends the information as a set of comma-separated values by means of SYSLOG transport. You can choose to use either the session-based or flow-based reporting format, depending on the level of granularity you need.

For example, a rule may collect session-based information about all audio-video traffic. You can specify how often to log the data and set the destination as an HSL server or pool.

Task Summary

Creating a publisher

Ensure that at least one destination associated with a pool of remote log servers exists on the BIG-IP system.
Create a publisher to specify where the BIG-IP system sends log messages for specific resources.
  1. On the Main tab, click System > Logs > Configuration > Log Publishers. The Log Publishers screen opens.
  2. Click Create.
  3. In the Name field, type a unique, identifiable name for this publisher.
  4. For the Destinations setting, in the Available list, select a destination, and click << to move the destination to the Selected list.
    Note: If you are using a formatted destination, select the destination that matches your log servers, such as Remote Syslog, Splunk, or ArcSight.
  5. Click Finished.

Creating a rule for high-speed logging

Before you can create a high-speed logging (HSL) rule, you need to create a publisher that defines the destination server or pool where the HSL logs are sent.
In an enforcement policy, a rule can specify that statistics about traffic affected by the rule are sent to an external high-speed logging server.
  1. On the Main tab, click Policy Enforcement > Policies. The Policies screen opens.
  2. Click the name of the enforcement policy you want to add rules to. The Properties screen for the policy opens.
  3. In the Policy Rules area, click Add. The New Rule screen opens.
  4. In the Name field, type a name for the rule.
  5. In the Precedence field, type an integer that indicates the precedence for the rule in relation to the other rules. Number 1 has the highest precedence. Rules with higher precedence are evaluated before other rules with lower precedence.
    Tip: It is a good idea to assign a different precedence value to each rule. That way, the action and order is perfectly clear, causing no possible conflicts. Also, if you start, for example, with 10 as the highest precedence, and leave space between the numbers assigned to other rules, you have space to add rules with higher or lower precedence than existing rules later.
  6. Use the Classification, Flow, and Custom Criteria tabs to identify the traffic that you want to be affected by this rule. Other tasks describe how to do this in detail.
  7. From the Reporting list, select Enabled.
  8. From the Report Granularity list, select the appropriate option:
    • To log details about subscribers and application sessions, select Session.
    • For more granular reporting of every TCP connection, select Flow.
  9. In the Volume Threshold field, specify, in octets, the amount of data to receive from the client, send to the client, and the total traffic volume before logging the information.
  10. In the Destination setting, for HSL, select the name of the publisher that specifies the server or pool of remote HSL servers to send the logs.
    Note: If you are using a formatted destination, select the publisher that matches your log servers, such as Remote Syslog, Splunk, or ArcSight.
  11. Click Finished.
You have created a rule that sends data about the traffic to external high-speed logging servers. The CSV reporting format differs depending on whether the report granularity is flow-based or session-based.

Session-based reporting format

In an enforcement policy, a rule can send session-based information about traffic that matches certain criteria to an external high-speed logging (HSL) server. The logs include the following comma-separated values in the order listed.

Field Description
Timestamp seconds The time the information was logged (along with the timestamp in milliseconds), specifies seconds using UNIX time format.
Timestamp msec The time the information was logged (along with the timestamp in seconds), specifies milliseconds using UNIX time format.
Report type The type of report. Always set to 3 for session-based reporting.
Subscriber ID A unique identifier (up to 64 characters) for the subscriber initiating the session, such as a phone number. The subscriber ID type determines the format.
Subscriber ID type The format of the subscriber ID. It can be E.164, IMSI, NAI, or Private.
3GPP parameters The list of 3GPP parameters, which can be imsi, imeisv, tower_id, or username.
Application ID A unique number that represents a particular application, and is used for classifying traffic.
Time since previous record sent The time, in seconds, since the last log entry was sent.
Flow end time The time the flow ended.
Bytes in The number of bytes received during this session.
Bytes out The number of bytes sent during this session.
Concurrent flows Always 0 (unsupported).
Opened flows Always 0 (unsupported).
Terminated flows Always 0 (unsupported).
Total transactions Always 0 (unsupported).
Successful transactions Always 0 (unsupported).
Application/category duration Summary of the duration of all flows for the session.
Reason The reason for sending the record. It can be 0 - reserved, 1 - volume threshold reached, 2- interval time, 3 - subscriber logout, or 4 - inactivity.

Example session-based reporting format

Oct 10 17:19:45 172.31.63.64 1349914925,546879,3,404234567123456,IMSI,linux,f501, 404234567123456,35827001,16394,1349914913,5469633,308908379, 0,0,0,0,0,5052,1 Oct 10 17:19:57 172.31.63.64 1349914937,546661,3,404234567123456,IMSI,linux,f501, 404234567123456,35827001,16394,1349914925,5550857,313317479, 0,0,0,0,0,5063,1 Oct 10 17:20:09 172.31.63.64 1349914949,546676,3,404234567123456,IMSI,linux,f501, 404234567123456,35827001,16394,1349914937,5636605,318053179, 0,0,0,0,0,5074,1

Flow-based reporting format

In an enforcement policy, a rule can send flow-based information about traffic that matches certain criteria to an external high-speed logging (HSL) server. The logs include the following comma-separated values in the order listed.

Field Description
Report time The time the information was logged in UNIX time format.
Report type The type of report; 0 – flow start, 1 – flow interim, 2 – flow end.
Subscriber ID A unique identifier (up to 64 characters) for the subscriber initiating the session, such as a phone number. The subscriber ID type determines the format.
Subscriber ID type The format of the subscriber ID. It can be E.164, IMSI, NAI, or Private.
Source IP address The source IP of the subscriber.
Source port The source port the subscriber.
Destination IP address The destination IP of the traffic.
Destination port The destination port for the traffic.
Protocol The protocol of the traffic for this flow, TCP or UDP.
Application ID A unique number that represents a particular application in this flow; it is used for classifying traffic.
Flow start time The time the flow started in UNIX time format.
Flow end time The time the flow ended in UNIX time format.
Bytes in The number of bytes received during this flow.
Bytes out The number of bytes sent during this flow.

Example flow-based reporting format

Sep 13 13:48:58 172.31.63.60 1347546777,654398,0,4086007577,E164,2001::10,52784,2001::2,80,6, 67,1347546774,628630,4278124286,4278124286,331,156 Sep 13 13:48:58 172.31.63.60 1347546777,654398,2,4086007577,E164,2001::10,52784,2001::2,80,6, 67,1347546774,628630,1347546775,382473,547,864
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)