After Secure Web Gateway (SWG) starts proxying web access, it provides information that you can use to monitor threats and to fine-tune URL filters.
On a BIG-IP® system with Access Policy Manager®, SWG can provide logs and reports.
On a BIG-IP system with an SWG subscription, SWG can provide overview statistics in addition to logs and reports.
You can view Secure Web Gateway (SWG) statistics on the BIG-IP® system and adjust URL filters to handle new threats based on the information that you gather from logs and reports.
Before you begin, event logging should be configured. SWG reports and charts depend on event logging for URL filters. For event logging to occur, log settings must be configured and then specified in the access profile, and a Category Lookup item must be run in the per-request policy.
The Secure Web Gateway (SWG) overview provides multiple reports and charts that summarize the top requests, such as top URLs, top categories by blocked request count, top users by permitted request count or by blocked request count, and so on. The overview can be customized to show the specific type of data that you are interested in.
In addition to the reports and charts on the overview, SWG provides the All Requests and Blocked Requests reports and charts. The reports can be filtered to show the information that you want to see.
Secure Web Gateway (SWG) is configured to produce reports and charts.
The system updates the statistics for charts and reports at five minute intervals: at five minutes after the hour, ten minutes after the hour, and so on.
Charts and data that you export from charts reflect the publishing interval of five minutes. For example, if you request data for the time period 12:40-13:40, the data in the chart or in the file that you export is for the time period 12:35-13:35. By default, the BIG-IP® system displays one hour of data.
Secure Web Gateway (SWG) reports and charts for weekly, monthly, and yearly time ranges include statistics up through the previously completed hour. The system performs hourly updates to the aggregated statistics.
Secure Web Gateway (SWG) reports display statistical information about web traffic on your system. These details are available:
You can configure the BIG-IP® system to log information about Access Policy Manager® (APM® ) and Secure Web Gateway events and send the log messages to remote high-speed log servers.
When configuring remote high-speed logging of events, it is helpful to understand the objects you need to create and why, as described here:
|Pool of remote log servers||Create a pool of remote log servers to which the BIG-IP system can send log messages.|
|Destination (unformatted)||Create a log destination of Remote High-Speed Log type that specifies a pool of remote log servers.|
|Destination (formatted)||If your remote log servers are the ArcSight, Splunk, or Remote Syslog type, create an additional log destination to format the logs in the required format and forward the logs to a remote high-speed log destination.|
|Publisher||Create a log publisher to send logs to a set of specified log destinations.|
|Log Setting||Add event logging for the APM system and configure log levels for it or add logging for URL filter events, or both. Settings include the specification of up to two log publishers: one for access system logging and one for URL request logging.|
|Access profile||Add log settings to the access profile. The log settings for the access profile control logging for the traffic that comes through the virtual server to which the access profile is assigned.|
Association of remote high-speed logging configuration objects
Access Policy Manager® (APM®) provides a default-log-setting. When you create an access profile, the default-log-setting is automatically assigned to it. The default-log-setting can be retained, removed, or replaced for the access profile. The default-log-setting is applied to user sessions only when it is assigned to an access profile.
Regardless of whether it is assigned to an access profile, the default-log-setting applies to APM processes that run outside of a user session. Specifically, on a BIG-IP® system with an SWG subscription, the default-log-setting applies to URL database updates.
Create a log destination of the Remote High-Speed Log type to specify that log messages are sent to a pool of remote log servers.
Create a formatted logging destination to specify that log messages are sent to a pool of remote log servers, such as Remote Syslog, Splunk, or ArcSight servers.
|Access Policy||Events that occur while an access policy runs.|
|Per-Request Policy||Events that occur while a per-request policy runs.|
|ACL||Events that occur while applying APM access control lists.|
|SSO||Events that occur during single-sign on.|
|Secure Web Gateway||Events that occur during URL categorization on a BIG-IP® system with an SWG subscription.|
|ECA||Events that occur during NTLM authentication for Microsoft Exchange clients.|
Log Allowed Events - When selected, user requests for allowed URLs are logged.
Event log levels are incremental, ranging from most severe (Emergency) to least severe (Debug). Setting an event log level to Warning for example, causes logging to occur for warning events, in addition to events for more severe log levels. The possible log levels, in order from highest to lowest severity are:
The table breaks a typical Access Policy Manager® (APM®) log entry into its component parts.
Feb 2 12:37:05 site1 notice tmm: 01490500:5: /Common/for_reports:Common: bab0ff52: New session from client IP 10.0.0.1 (ST=/CC=/C=) at VIP 220.127.116.11 Listener /Common/site1_http (Reputation=Unknown)
|Information Type||Example Value||Description|
|Timestamp||Feb 2 12:37:05||The time and date that the system logged the event message.|
|Host name||site1||The host name of the system that logged the event message. Because this is typically the host name of the local machine, the appearance of a remote host name could be of interest.|
The text value of the log level for the message.
The process that generated the event.
|PID||||The process ID.|
|Log ID||01490500||A code that signifies the product, a subset of the product, and a message number.|
|Level||5||The numeric value of the log level for the message.|
|Partition||/Common/for_reports:Common||The partition.to which configuration objects belong.|
|Session ID||bab0ff52||The ID associated with the user session.|
|Log message||New session from client IP 10.0.0.1 (ST=/CC=/C=) at VIP 18.104.22.168 Listener /Common/site1_http (Reputation=Unknown)||The generated message text.|
The BIG-IP® system provides two local logging destinations:
The BIG-IP system provides a default log publisher for local logging, sys-db-access-publisher; initially, it is configured to publish to the local-db destination and the local-syslog destination. Users can create other log publishers for local logging.
The BIG-IP® system publishes logs for portal access traffic and for connections to virtual desktops (VDI) to the /var/log/rewrite* files. APM® cannot publish these logs to remote destinations.
APM can publish URL request logs to remote or local destinations. Logs published to the local-db destination are stored in the local database and are available for display from the Configuration utility. Logs published to the local-syslog destination are stored in the /var/log/urlfilter.log file.
APM can publish access system logs to remote or local destinations. Logs published to the local-db destination are stored in the local database. Logs in the local database are available for display in APM reports. Logs published to the local-syslog destination are stored in the /var/log/apm file.
The BIG-IP® system log messages contain codes that provide information about the system. You can run the Linux command cat log |bigcodes |less at the command prompt to expand the codes in log messages to provide more information. For example:
Jun 14 14:28:03 sccp bcm56xxd [ 226 ] : 012c0012 : (Product=BIGIP Subset=BCM565XXD) : 6: 4.1 rx [ OK 171009 Bad 0 ] tx [ OK 171014 Bad 0 ]
Event log duplication
The figure illustrates a configuration that writes duplicate logs. Two log publishers specify the same log destination, local-db. Each log publisher is specified in one of the log settings that are assigned to an access profile. Logs are written to the local-db destination twice.
Duplicate log messages are written when the same log destination is specified by two or more log publishers and more than one of the log publishers is specified in the log settings that are assigned to an access profile.
One way to avoid or eliminate this problem is to specify only one log setting for each access profile. Another is to ensure that the log publishers you associate with log settings for an access profile do not contain duplication log destinations.
Log levels can be configured in various ways that depend on the specific functionality. Log levels for access portal traffic and for connections to virtual desktops are configured in the System area of the product. The log level for the URL database download is configured in the default-log-setting in the Access Policy Event Logs area of the product. The log level for NTLM authentication of Microsoft Exchange clients is configured using the ECA option in any log setting. Other access policy (and Secure Web Gateway) log levels are configured in any log setting.
Change the logging level for access policy events when you need to increase or decrease the minimum severity level at which Access Policy Manager® (APM®) logs that type of event. Follow these steps to change the log level for events that are related to portal access traffic or related to connections to virtual desktops (VDI).