Applies To:

Show Versions Show Versions

Manual Chapter: Policy Management Guide for the BIG-IP WebAccelerator Module: 14 - Specifying Log Formats
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>


14

Specifying Log Formats


Defining logging options

Many sites perform traffic analysis against the log files generated by their web servers, and some sites also use custom scripts to manipulate these log files in various ways.

The WebAccelerator system creates a log file called hit logs to log cache hits and misses. These hit logs contain the same sort of information that your web servers create in their log files. For user-defined acceleration policies, you can tailor the information that appears in the WebAccelerator system's hit logs so they work seamlessly with whatever analysis tools you use for your web server log files.

Note

By default, logging is disabled for the pre-defined acceleration policies. You can modify log formats only for acceleration policies that you create. To enable logging, you must first create a user-defined custom acceleration policy. For more information, see Creating user-defined acceleration policies , located in Chapter 3.

From the Log Format screen, you can specify what the WebAccelerator system should log, and the format in which it should log that information. You can separately log HTTP and HTTPS requests and for each, you can select from the following logging options:

  • Log all transactions.
  • Log only those transactions that the WebAccelerator system serviced from cache.
  • Log nothing.
Note

If you log HTTP and HTTPS requests separately, you can customize the WebAccelerator system's logging behavior for each protocol individually. For example, you could describe a specific log file format for HTTP requests while not logging HTTPS requests at all.

Specifying a log format

Log headers are the lines that appear at the top of a log file, and are used to identify the type and order of the information written to each line in the log file. Some log analysis software uses log headers to determine how to parse the log file.

For hit log files, you can specify the format for the hit log files by choosing from a selection of pre-defined log formats. (See To specify a log format .) The pre-defined log formats represent a collection of logging information that is commonly used by web servers.

The pre-defined log formats consist of:

  • NCSA Common (no header)
  • NCSA Common (Netscape header)
  • NCSA Combined (no header)
  • NCSA Combined (Netscape header)
  • W3C Extended

The three common conventions for log headers are:

  • No header line
    ApacheTM web servers use this option. By default, Apache web servers write access logs in a format that are identical to the NCSA Common format.
  • NCSA Common or Combined headers
    Netscape® servers, and their descendants (such as the iPlanetTM Enterprise Server) write a header line that is unique to these family of servers. These servers generally use either the NCSA Common or Combined log format. The header lines are comprised of keywords. For example:
  • #format=%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] ....

  • W3C headers
    Most Microsoft® Internet Information Services (IIS) web servers write log files in the extended log file format, which are defined by a W3C working draft. For more information, see http://www.w3.org/TR/WD-logfile.html.

Or if you require a log header and the header line that your web server writes does not conform to any of these pre-defined log format conventions, you can define a custom log format. For more information, see Creating custom log formats .

NCSA Common log format

The NCSA Common log format syntax is as follows:

host rfc931 username [date:time UTC_offset] "method URI?query_parameters protocol" status bytes

For example:

125.125.125.2 - - [10/Oct/2002:23:44:03 -0600]
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045

NCSA Combined log format

The NCSA Combined log format syntax is as follows:

host rfc931 username [date:time UTC_offset] "method URI?query_parameters protocol" status bytes "referrer" "user_agent" "cookie"

For example:

125.125.125.2 - - [23/Oct/2002:23:44:03 -0600]
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045
"http://www.siterequest.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" "UserID=ssarette;Category=PDA;Selection=Various"

W3C Extended log format

The W3C extended log format syntax is as follows:

date time rfc931 username host method URI query_parameters status bytes request_length time_taken protocol user_agent cookie referrer

For example:

2002-10-23 23:44:03 205.47.62.112 - 125.125.125.2 GET /apps/example.jsp sessionID=34h76 200 3045 124 138 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0 UserID=ssarette;Category=PDA;Selection=Various http://www.siterequest.com

To specify a log format

  1. On the Main tab of the navigation pane, click Policies.
    The Policies screen opens, displaying a list of acceleration policies.
  2. Click Edit next to the acceleration policy that you want to edit.
    The Policy Editor screen opens.
  3. From the function bar (located at top left of the Policy Editor screen, next to the F5 logo), click Log Format.
    The Log Format screen opens.
  4. If you want the WebAccelerator system to create individual logs for the HTTP and HTTPS protocols, check the box for Log HTTP and HTTPS requests separately in the What to Log area.
    The screen refreshes with additional HTTP Log options.
  5. In the HTTP Log area, click the button next to the following options as required:
    • Log all transactions
    • Only log transactions served from cache
    • Do not log
  6. If you select Log all transactions, or Only log transactions served from cache, then select a format for the HTTP logs from the Log Format list.
  7. In the HTTPS Log area (available only if you are logging HTTP and HTTPS separately), click the button next to the following options as required:
    • Log all transactions
    • Only log transactions served from cache
    • Do not log
  8. If you selected Log all transactions, or Only log transactions served from cache, then select a format for the HTTPS logs from the Log Format list.
  9. Click the Save button.

Creating custom log formats

The WebAccelerator system also provides you with the option to create customized formats that log additional information not available using a pre-defined log format.

To create a customized log format

  1. On the Main tab of the navigation pane, click Policies.
    The Policies screen opens, displaying a list of acceleration policies.
  2. Click Edit next to the acceleration policy that you want to edit.
    The Policy Editor screen opens.
  3. From the function bar (located at top left of the Policy Editor screen, next to the F5 logo), click Log Format.
    The Log Format screen opens.
  4. If you want the WebAccelerator system to create individual logs for the HTTP and HTTPS protocols, check the box for Log HTTP and HTTPS requests separately in the What to Log area.
    The screen refreshes with additional HTTP Log options.
  5. In the HTTP Log section, click the button next to the following options as required:
    • Log all transactions
    • Only log transactions served from cache
  6. Next to Log Format list, click the New button.
    The custom format screen opens.
  7. In the Name box, type a name for the custom log you are creating.
  8. From the Header Format Type list, select a format.
  9. From the Format Text list located in the blue area on the right side of the screen, select the individual components for the Header Format, and click the Add button.
    The Log Item Format box fills with the corresponding item.
  10. Click the Save button.
  11. From the Log Format list, select the custom format you created.
  12. In the HTTPS Log area (available only if you are logging HTTP and HTTPS separately), click the button next to the following options as required:
    • Log all transactions
    • Only log transactions served from cache
  13. From the Log Format list, select a format for the HTTPS logs.
  14. Click the Save button.

Configuring a customized log format example

This section of the chapter provides information about how to configure an example customized log format. For this example, you want your hit logs to contain the following information:

  • IP of the requesting client
  • Date and time of the request
  • HTTP method used for the request
  • Query URI
  • Query parameters passed using the GET method
  • HTTP version used by the request
  • HTTP status returned on the response
  • Length of the response
  • Actual origin server to which the WebAccelerator system sends required requests

Your origin servers are all running Apache, so you want your log lines to follow the NCSA Common (No Header) formatting convention, as follows:

xxx.xxx.xxx.xxx [date:time UTC_offset] "Method uri?query_parameter HTTP_ver" response_status response_length

However, you want one to customize the NCSA Common (no header) log format to add an additional field that includes the origin server hostname and IP address.

To customize the NCSA Common (no header) log format with the origin server information

  1. On the Main tab of the navigation pane, click Policies.
    The Policies screen opens, displaying a list of acceleration policies.
  2. Click Edit next to the acceleration policy that you want to edit.
    The Policy Editor screen opens.
  3. From the function bar (located at top left of the Policy Editor screen next to the F5 logo), click Log Format.
    The log format screen opens.
  4. In the Logs area, click the Log all transactions button.
  5. From the Log Format list, select NCSA Common (no header) and click the Edit button.
    The edit log format screen opens.
  6. In the Name box, type a meaningful name for the log.
  7. From the Format Text list located in the blue area on the right side of the screen, select Hostname/IP of Origin Server and click the Add button.
    The %B symbol appears in the Log Item Format box. Verify that there is a space before the formatting code in the box. If there is not, manually add a space before the formatting code.
  8. Click the Save button.
    The log format screen opens with the new format displayed in the Log Format menu.
  9. Click the Save button.

For a new or modified acceleration policy to be in effect for your site, you must publish it. See Publishing acceleration policies , located in Chapter 3, for more information.




Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)