Manual Chapter : Using the Request Logging Profile

Applies To:

Show Versions Show Versions

BIG-IP AAM

  • 11.5.10, 11.5.9, 11.5.8, 11.5.7, 11.5.6, 11.5.5, 11.5.4, 11.5.3, 11.5.2, 11.5.1
Manual Chapter

Using the Request Logging Profile

Overview: Configuring a Request Logging profile

The Request Logging profile gives you the ability to configure data within a log file for HTTP requests and responses, in accordance with specified parameters.

Task summary

Perform these tasks to log HTTP request and response data.

About the Request Logging profile

Many sites perform traffic analysis against the HTTP log files that their web servers generate. With the Request Logging profile, you can specify the data and the format for HTTP requests and responses that you want to include in a log file. If you prefer, you can tailor the information that appears in the logs so that the logs work seamlessly with whatever analysis tools you use for your origin web server's HTTP log files.

Standard log formats

Log headers appear in the lines at the top of a log file. You can use log headers to identify the type and order of the information written to each line in the log file. Some log analysis software also uses log headers to determine how to parse a log file.

There are three common conventions for log headers shown here.

Convention Description
No header line Apache™ web servers use this option. By default, Apache web servers write access logs in a format that is identical to the NCSA Common format.
NCSA Common or Combined headers Netscape® servers, and their descendants (such as the iPlanet™ Enterprise Server) write a log header line that is unique to this family of servers. These servers generally use either the NCSA Common or Combined log format, and the log header lines are composed of keywords. For example:
#format=%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] ....
W3C headers Most Microsoft® Internet Information Services (IIS) web servers write log files in the extended log file format, which is defined by a W3C working draft.

The logging information that is commonly used by origin web servers consists of the following conventions:

  • NCSA Common (no log header)
  • NCSA Common (Netscape log header)
  • NCSA Combined (no log header)
  • NCSA Combined (Netscape log header)
  • W3C Extended

NCSA Common log format example

This is the NCSA Common log format syntax:

host rfc931 username [date:time UTC_offset] 
"method URI?query_parameters protocol" status bytes

Here is an example that uses this syntax:

125.125.125.2 - - [03/Apr/2011:23:44:03 -0600] 
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045

NCSA Combined log format example

This is the NCSA Combined log format syntax:

host rfc931 username [date:time UTC_offset] 
"method URI?query_parameters protocol" status bytes 
"referrer" "user_agent" "cookie"

Here is an example that uses this syntax:

125.125.125.2 - - [03/Apr/2011:23:44:03 -0600] 
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045
"http://www.siterequest.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" 
"UserID=ssarette;Category=PDA;Selection=Various"

W3C Extended log format example

This is the W3C extended log format syntax:

date time rfc931 username host method URI query_parameters 
status bytes request_length time_taken protocol user_agent cookie referrer 

Following is an example that uses this syntax:

2011-04-03 23:44:03 205.47.62.112 - 125.125.125.2 
GET /apps/example.jsp sessionID=34h76 200 3045 124 138 
HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0 
UserID=ssarette;Category=PDA;Selection=Various http://www.siterequest.com

Request Logging profile settings

With the Request Logging profile, you can specify the data and the format for HTTP requests and responses that you want to include in a log file.

General Properties

Setting Value Description
Name No default Specifies the name of the profile.
Parent Profile Selected predefined or user-defined profile Specifies the selected predefined or user-defined profile.

Request Settings

Setting Value Description
Request Logging Disabled Enables logging for requests.
Template   Specifies the directives and entries to be logged.
HSL Protocol UDP Specifies the protocol to be used for high-speed logging of requests.
Pool Name None Defines the pool associated with the virtual server that is logged.
Respond On Error Disabled Enables the ability to respond when an error occurs.
Error Response None Specifies the response text to be used when an error occurs.
For example, the following response text provides content for a 503 error.
<html>
 <head>
  <title>ERROR</title>
 </head>
 <body>
  <p>503 ERROR-Service Unavailable</p>
 </body>
</html>
            
Close On Error Disabled When enabled, and logging fails, drops the request and closes the connection.
Log Logging Errors Disabled Enables the ability to log any errors when logging requests.
Error Template None Defines the format for requests in an error log.
HSL Error Protocol UDP Defines the protocol to be used for high-speed logging of request errors.
Error Pool Name None Specifies the name of the error logging pool for requests.

Response Settings

Setting Value Description
Response Logging Disabled Enables logging for responses.
Log By Default Enabled Defines whether to log the specified settings for responses by default.
Template None Specifies the directives and entries to be logged.
HSL Protocol UDP Specifies the protocol to be used for high-speed logging of responses.
Pool Name None Defines the pool name associated with the virtual server that is logged.
Log Logging Errors Disabled Enables the ability to log any errors when logging responses.
Error Template None Defines the format for responses in an error log.
HSL Error Protocol UDP Defines the protocol to be used for high-speed logging of response errors.
Error Pool Name None Specifies the name of the error logging pool for responses.

Request Logging parameters

This table lists all available parameters from which you can create a custom HTTP Request Logging profile. These are used to specify entries for the Template and Error Template settings For each parameter, the system writes to the log the information described in the right column.

Table 1. Request logging parameters
Parameter Log file entry description
BIGIP_BLADE_ID An entry for the slot number of the blade that handled the request.
BIGIP_CACHED An entry of Cached status: true, if the response came from BIG-IP® cache, or Cached status: false, if the response came from the server.
BIGIP_HOSTNAME An entry for the configured host name of the unit or chassis.
CLIENT_IP An entry for the IP address of a client, for example, 192.168.74.164.
CLIENT_PORT An entry for the port of a client, for example, 80.
DATE_D A two-character entry for the day of the month, ranging from 1 (note the leading space) through 31.
DATE_DAY An entry that spells out the name of the day.
DATE_DD A two-digit entry for the day of the month, ranging from 01 through 31.
DATE_DY A three-letter entry for the day, for example, Mon.
DATE_HTTP A date and time entry in an HTTP format, for example, Tue, 5 Apr 2011 02:15:31 GMT.
DATE_MM A two-digit month entry, ranging from 01 through 12.
DATE_MON A three-letter abbreviation for a month entry, for example, APR.
DATE_MONTH An entry that spells out the name of the month.
DATE_NCSA A date and time entry in an NCSA format, for example, dd/mm/yy:hh:mm:ss ZNE.
DATE_YY A two-digit year entry, ranging from 00 through 99.
DATE_YYYY A four-digit year entry.
HTTP_CLASS The name of the httpclass profile that matched the request, or an empty entry if a profile name is not associated with the request.
HTTP_KEEPALIVE A flag summarizing the HTTP1.1 keep-alive status for the request:: aY if the HTTP1.1 keep-alive header was sent, or an empty entry if not.
HTTP_METHOD An entry that defines the HTTP method, for example, GET, PUT, HEAD, POST, DELETE, TRACE, or CONNECT.
HTTP_PATH An entry that defines the HTTP path.
HTTP_QUERY The text following the first ? in the URI.
HTTP_REQUEST The complete text of the request, for example, $METHOD $URI $VERSION.
HTTP_STATCODE The numerical response status code, that is, the status response code excluding subsequent text.
HTTP_STATUS The complete status response, that is, the number appended with any subsequent text.
HTTP_URI An entry for the URI of the request.
HTTP_VERSION An entry that defines the HTTP version.
NCSA_COMBINED An NCSA Combined formatted log string, for example, $NCSA_COMMON $Referer ${User-agent} $Cookie.
NCSA_COMMON An NCSA Common formatted log string, for example, $CLIENT_IP - - $DATE_NCSA $HTTP_REQUEST $HTTP_STATCODE $RESPONSE_SIZE.
RESPONSE_MSECS The elapsed time in milliseconds (ms) between receiving the request and sending the response.
RESPONSE_SIZE An entry for the size of response in bytes.
RESPONSE_USECS The elapsed time in microseconds (µs) between receiving the request and sending the response.
SERVER_IP An entry for the IP address of a server, for example, 10.10.0.1.
SERVER_PORT An entry for the port of a server, for example, 80.
SNAT_IP An entry for the self IP address of the BIG-IP-originated connection to the server when SNAT is enabled, or an entry for the client IP address when SNAT is not enabled.
SNAT_PORT An entry for the port of the BIG-IP-originated connection to the server when SNAT is enabled, or an entry for the client port when SNAT is not enabled.
TIME_AMPM A twelve-hour request-time qualifier, for example, AM or PM.
TIME_H12 A compact twelve-hour time entry for request-time hours, ranging from 1 through 12.
TIME_HRS A twelve-hour time entry for hours, for example, 12 AM.
TIME_HH12 A twelve hour entry for request-time hours, ranging from 01 through 12.
TIME_HMS An entry for a compact request time of H:M:S, for example, 12:10:49.
TIME_HH24 A twenty-four hour entry for request-time hours, ranging from 00 through 23.
TIME_MM A two-digit entry for minutes, ranging from 00 through 59.
TIME_MSECS An entry for the request-time fraction in milliseconds (ms).
TIME_OFFSET An entry for the time zone, offset in hours from GMT, for example, -11.
TIME_SS A two-digit entry for seconds, ranging from 00 through 59.
TIME_UNIX A UNIX time entry for the number of seconds since the UNIX epoch, for example, 00:00:00 UTC, January 1st, 1970.
TIME_USECS An entry for the request-time fraction in microseconds (µs).
TIME_ZONE An entry for the current Olson database or tz database three-character time zone, for example, PDT.
VIRTUAL_IP An entry for the IP address of a virtual server, for example, 192.168.10.1.
VIRTUAL_NAME An entry for the name of a virtual server.
VIRTUAL_POOL_NAME An entry for the name of the pool containing the responding server.
VIRTUAL_PORT An entry for the port of a virtual server, for example, 80.
VIRTUAL_SNATPOOL_NAME The name of the Secure Network Address Translation pool associated with the virtual server.
WAM_APPLICATION_NAM An entry that defines the name of the BIG-IP® acceleration application that processed the request.
WAM_X_WA_INFO An entry that specifies a diagnostic string (X-WA-Info header) used by BIG-IP acceleration to process the request.
NULL Undelineated strings return the value of the respective header.