Applies To:

Show Versions Show Versions

Manual Chapter: Troubleshooting
Manual Chapter
Table of Contents   |   << Previous Chapter

BIG-IP® Access Policy Manager® provides ways to troubleshoot issues that you may encounter from time to time. There are a number of files, utilities, and command line interfaces that you can use to pinpoint the problem areas and resolve them quickly.
This appendix provides several different examples that you can refer to in order to understand how Access Policy Manager troubleshooting tools work. Following the examples, you will find sections on Access Policy Manager log messages and Kerberos error messages.
You can find all log messages relating to network access in the Configuration utility. On the navigation pane, expand System, click Logs, and select Access Control. However, you view ACL-related log messages in a different location: in the navigation pane, expand System, click Logs, and select Packet Filter.
Notice. This level provides the most basic logging information about users attempts to establish a network connection. Within the log, you can track a users access by his session ID, as shown in Figure E.1.
Informational. This level provides more in-depth logging information about user access. We recommend you use this level for analyzing access issues on user logon failures.
By default, the log level is set to Notice. This example shows you how to change the default log level to Informational.
2.
On the navigation pane, expand System, and click Logs.
The Logs screen opens.
3.
On the menu bar, click Options.
The Local Traffic Logging screen opens.
4.
Scroll down to the Access Control Logging area; for the Access Control setting, select Informational.
5.
Under Secure Connectivity, for the Network Access setting, select Informational.
6.
Click Update.
For this example, disable your Microsoft Windows® firewall setting on the client operating system, for instance, Windows XP. You must set up an access policy where the client checks for anti-virus software. When you attempt to access the virtual server, your access request fails because the Windows firewall setting is disabled.
You can now examine the logs displayed on the access control log menu. The system generates a series of log messages as a result of this failure.
Tip: Make sure the log messages are displayed in chronological order, from the most recent logs to the older ones. Within the Log message screen, click TimeStamp to sort the logs based on the most recent times.
Figure E.2 displays a sample log message. The most pertinent data is highlighted in the figure, and described, following.
Note: The session variable displayed in figure includes windows_check_fw; this phrase is no longer used in the session variable name. It has been replaced by check_fw.
check_fw. This is the session variable object that represents the endpoint security check on a firewall. This variable is allocated if your access policy profile has a firewall action included in your endpoint security check.
state. This is the objects attribute that describes the status of the Windows firewall running on your clients desktop.
0 value. This value means that the current state of the Windows firewall is disabled. If the value displayed is 1, the Windows firewall is then enabled.
Since the firewall check returned a result of 0, the final return value on the access policy check resulted in an access denied policy ending. Therefore, the sessionID created for your access is immediately deleted.
This example shows log messages displayed if the system encounters a problem with authentication. Assuming that the user passed the endpoint security check, the logon screen appears, requesting valid credentials. For the purpose of this example, enter an invalid credential at the logon page. As a result of inputting incorrect credentials, the authentication fails on your authentication server, and you are directed to a logon denied page.
Figure E.3 displays sample log messages, showing the failure within an Microsoft Active Directory® server.
The example in figure E.3 displays the highlighted response received from the Active Directory server, which states that the user name entered on the logon page does not appear to be a valid user in the Active Directory database.
You can use the adminreporting utility feature of Access Policy Manager to view logon reports.To run this utility, use SSH to log on to the system and type the following command: adminreport.pl -logon logs. This command provides a summary of logon reports based on the logs in the var/log/firepass file.
Figure E.4 displays a summary of a logon report based on logs generated to the /var/log/FirePass file. For a list of all the commands available for this utility, refer to Chapter 17, Logging and Reporting. Alternatively, you can view the same summary report by using the navigation pane. Expand Overview, and click Reports, then on the Reports screen, on the menu bar, click All Sessions.
Access Policy Manager provides a tool called logging action, within the visual policy editor. This tool lets you tailor the logging of any session variables to the access control logs, so that you can better identify and understand the cause of a users logon failure.
Figure E.5 displays a sample log message generated based on a logon failure. You can view this message by using the navigation pane. Expand System, click Logs, and on the menu bar, click Access Control.
1.
On the navigation pane, expand Overview, and click Reports.
The Reports screen opens.
2.
Figure E.6 displays a sample report, showing logon history.
Table E.1 lists all log messages from the BIG-IP® Access Policy Manager®.
Log level
00000000: Number of ports should not exceed: <Port Count>
Specifies that the APD daemon started with the wrong parameters. This can happen only if the administrative user modifies the start scripts for APD.
Make sure that the command line arguments to the APD daemon have not been modified in the /etc/bigstart/scripts/apd file. Factory settings:
-d 3 -f
00000000: Number of threads should not exceed: <Thread Count>
Specifies that the APD daemon started with the wrong parameters. This can happen only if the administrative user modifies the start scripts for APD.
Make sure that the command line arguments to the APD daemon have not been modified in the /etc/bigstart/scripts/apd file. Factory settings:
-d 3 -f
00000000: Couldn't create APD listener: <Listener ID>
Specifies that the APD daemon started with the wrong parameters. This can happen only if the administrative user modifies the start scripts for APD.
Make sure that the command line arguments to the APD daemon have not been modified in the /etc/bigstart/scripts/apd file. Factory settings:
-d 3 -f
<Session ID> Executed agent '%s', return value %d
Specifies the name of the agent that is started and the returned value. The returned value is an integer.
<Session ID> Following rule '%s' from item '%s' to ending '%s'
Indicates the access policy items that the user system followed to reach the specified ending. The name of the ending is ending_denied, webtop or redirect ending.
<Session ID> Following rule '%s' from item '%s' to item '%s'
Specifies the rules that are followed when the system processes the access policy.
Session variable <Session Variable Name> set to <value>
This is an informational message that the variable <Session Variable Name> is set to the value <value>, and the access policy can use it in the session.
<Session ID> Connectivity resource '%s' assigned through resource group '%s'
Specifies that the resource assign action has assigned the specified connectivity resource to the session.
<Session ID> ACL '%s' assigned
Specifies that the resource assign action has assigned the specified ACL to the session.
<Session ID> Username '%s'
Specifies the user name used for the logon page.
<Session ID>: agent: Retrieving AAA server: <ServerName>
Specifies that the AAA agent is retrieving the AAA server information.
<Session ID>: agent: No AAA server associated with <Agent Name>
Specifies that the access policy configuration is incomplete. The AAA agent specified in the log message is not associated with a valid AAA server.
Make sure a AAA Server is assigned in the AAA action <Agent Name> configuration in the access policy.
<Session ID>: agent: Failed to decrypt <StringName> of AAA server: <Server Name>
Specifies that APD daemon failed to initialize the access policy. This error indicates that the APD daemon is unable to decrypt the administrative password for the AAA server specified in the log message. This indicates a critical system failure.
No troubleshooting information available.
<Session ID>: agent: Unknown agent type <TypeID>
Specifies that the APD daemon failed to initialize the access policy. The access policy contains an agent of unknown type. This indicates a critical system failure.
No troubleshooting information available.
<Session ID> AD agent: Auth (logon attempt:<Count>): authenticate with '<UserName>' <Result>
Informational. Specifies the <Result> of an Active Directory authentication attempt. The result is either failed or successful.
<Session ID> AD agent: Query: query with '<Filter>' <Result>
Informational. Specifies the <Result> of an Active Directory query attempt. The result is either failed or successful.
<Session ID>: agent: ERROR: <ErrorMessage>
Specifies that one of the access policy agents encountered an error, as described by the error message, during access policy processing.
No troubleshooting information available.
<Session ID>: agent: EXCEPTION: <ExceptionMessage>
Specifies that one of the access policy agents encountered an error, as described by the error message, during access policy processing.
No troubleshooting information available.
<Session ID> <AuthType> module: ERROR: <ErrorMessage>
Specifies that a AAA server operation of the type specified in the log message failed with the error described by the error message.
<AuthType> indicates the authentication module in which the error occurred. The <ErrorMessage> contains information that can point to the cause of the error.
<Session ID> <AuthType> module: EXCEPTION: <ExceptionMessage>
Specifies that a AAA server operation of the type specified in the log message failed with the error described by the error message.
<AuthType> indicates the authentication module in which the error occurred. The <ExceptionMessage> contains information that can point to the cause of the error.
<Session ID> LDAP agent: Auth (logon attempt:<Count>): authenticate with '<UserName>' <Result>
Provides an informational message that indicates that the LDAP authentication attempt occurred. The Result is either failed or successful.
<Session ID> LDAP agent: Query: query <Result>, dn: <DN>, filter: <Filter>
Provides an informational message that indicates that the LDAP query attempt occurred. The Result is either failed or successful.
<Session ID> <AuthType> module: ERROR: ldap_unbind() failed, <ErrorMessage>
Specifies that the LDAP unbind operation for either LDAP or Active Directory® failed with the error described in the error message.
<AuthType> indicates the authentication module in which the error occurred. The <ErrorMessage> for ldap_unbind() contains more information about the cause of the error.
<Session ID> RADIUS agent: (logon attempt:<Count>) authenticate with <UserName>' <Result>
Specifies an informational message that indicates that the RADIUS authentication attempt occurred. The Result is either failed or successful.
<Session ID> RADIUS agent: (logon attempt:<Count>) radius challenge response received, reply-message: <Message>
00000000: AD agent: ERROR: %s failed for <hostname/IPaddr>
Specifies that the Active Directory action encountered an error while trying to authenticate against the external AAA server with the host name and IP address listed in the error message.
Make sure that DNS is properly configured to resolve the forward and reverse lookup for the AAA server.
<Session ID> AD agent: Auth (logon attempt: <Count> ): password changed successfully for '<UserName>'
<Session ID> AD agent: Auth (logon attempt: <Count>): Domain password has been expired and must be changed for '<UserName>'
<Session ID> AD agent: Auth (logon attempt: <Count>): failed to change password for '<UserName>'
00000000: Access policy '%s' configuration has changed. Access profile '%s' configuration changes need to be applied for the new configuration
Specifies that the access policy configuration has changed.
The modified or new configuration changes are not yet active and you must activate the access policy for the changes to take effect.
00000000: ERROR: Session db interface layer internal error: %d.
Specifies that the APD daemon failed to communicated with the session database. This indicates a critical system failure.
No troubleshooting information available.
<Session ID> Agent execution failed for agent: %d and access policy item: %d
Specifies that an access policy action encountered an error, described in the error message, while the access policy was processing.
No troubleshooting information available.
<Session ID> Invalid rule exists in access policy. Unable to find nextnode.
Specifies that the access policy configuration is not valid. One of the access policy rules is followed by an item that is not valid.
No troubleshooting information available.
00000000: Request from remote client could not be received from socket. Socket error: %s
Specifies that an error occurred while the system was receiving data from the remote client during access policy processing. Indicates a critical system failure.
No troubleshooting information available.
<Session ID> Access Policy execution failed with error: %d
Specifies that, during access policy processing, an access policy action encountered an error, described in the error message.
No troubleshooting information available.
<Session ID> Response could not be sent to remote client. Socket error:%s
Specifies that an error, described in the error message, occurred while sending the data response to the remote client during access policy processing. This might occur if the remote client disconnects during access policy processing.
No troubleshooting information available.
<Session ID> Rule evaluation failed with error: %s
Specifies that the error described in the error message occurred while trying to evaluate an access policy rule during access policy processing.
No troubleshooting information available.
<Session ID> Invalid session variable exists in rule expression.
Specifies that an error occurred while attempting to evaluate an access policy rule during access policy processing.
This error indicates that a session variable that is not valid is present in the rule expression.
Make sure that the session variable configured in the access policy rule does exist when the rule runs.
<Session ID> Unable to find session variable used in rule expression.
Specifies that an error occurred while attempting to evaluate an access policy rule during access policy processing.
This error indicates that a session variable that is not valid is present in the rule expression.
Make sure that the session variable configured in the access policy rule does exist when the rule runs.
00000000: Configuration change notification received for an unknown access profile: %s
STOP Specifies that the APD has received a configuration change notification for an unknown access profile.
No troubleshooting information available.
00000000: Configuration add notification received for an already existing profile: %s
Specifies that the APD has received ADD notification for an existing access profile.
No troubleshooting information available.
00000000: Invalid request header received from remote client. Socket error: %s
Specifies that the response received during access policy processing from a remote client is not valid.
The log message logs the incoming HTTP request header received from the remote client.
No troubleshooting information available.
00000000: Invalid POST request received from remote client. Len: %d
Specifies that the response received during access policy processing from the remote client is not valid.
The log message logs the length of the incoming HTTP POST request received from the remote client.
No troubleshooting information available.
00000000: Request header parsing failed while processing request from remote client
Specifies that an error occurred while processing the received request from the remote client during access policy processing.
No troubleshooting information available.
<Session ID> Couldn't get session variable from session db. Session var: %s
Specifies that APD failed to retrieve a session variable (logged by the log message) from the session database.
No troubleshooting information available.
<Session ID> File Check Agent: File check failed.
Specifies that the file check action encountered an error during access policy processing.
Log and inspect the session variables for the file check action.
00000000: A new access profile: %s has been initialized
Specifies that the system has initialized the specified access profile.
Access Policy Manager accepts any request received for this access profile from this point forward, and sends these requests through the associated access policy.
00000000: A new access policy: %s has been initialized
Specifies that the system has initialized a new access policy.
00000000: Access profile: %s has been removed.
Specifies that the system has deleted an access profile.
Access Policy Manager denies any request received for this access profile from this point forward.
00000000: Access policy: %s has been removed.
Specifies that the system has deleted an access profile.
00000000: Access profile: %s configuration changes need to be applied for the new configuration to take effect.
Specifies that the system has detected changes you have made to the access profile configuration.
The modified or new configuration changes are not yet active. You must activate the access policy for the new changes to take effect.
00000000: Access profile: %s configuration has been applied. Newly active generation count is: %d
Specifies that the system has started the access policy associated with the access profile.
Access Policy Manager increments the generation count by one every time an access policy is activated.
<Session ID> Access policy result: %s
The final result of the access policy. Valid results are Logon_Denied or Webtop
<Session ID> Retry Username '<UserName>'
00000000: <Session ID> Failed to store configuration variable (error:%d, name:'%s', value:'%s')
Specifies that APD failed to store a session variable (logged by the log message) in the session database.
The log message logs the name of the error encountered along with the variable and value of the variable.
Access Policy Manager was unable to store the session variable in the session database. Either an internal processing error or a failure in database memory allocation occurred.
<Session ID> <AuthType> agent: No AAA server associated with <ServerName>.
Specifies that the AAA action encountered an error during access policy processing, because the AAA server information could not be located.
Make sure that the AAA Server <ServerName> exists in the bigip.conf file. This might happen when a AAA server is deleted from bigip.conf, but the AAA server is still being used by a AAA action.
<Session ID> AD module: WARNING: <Action> <Object> failed in <FunctionName>(): <ErrorMessage> (ErrorCode)
Specifies that the Active Directory Auth or Query action encountered an error during access policy processing.
Action has one of the values:
- query with
- authentication with
- change password for
Object has one of the values:
- Filter
- <AdminUserName>
- <UserName>
The error message is included with the source code function name.
Refer to the <ErrorMessage> text, which contains information about the cause of the error.
<Session ID> AD module: ERROR: <Action> <Object> failed in <FunctionName>(): <ErrorMessage> (ErrorCode)
Specifies that the Active Directory Auth or Query action encountered an error during access policy processing.
Action has one of the values:
- query with
- authentication with
- change password for
Object has one of the values:
- Filter
- <AdminUserName>
- <UserName>
The error message is included with the source code function name.
Refer to the <ErrorMessage> text, which contains information about the cause of the error.
<Session ID> RADIUS module: ERROR: authentication with <UserName> failed in <FunctionName>(): <ErrorMessage> (ErrorCode)
Specifies that, during access policy processing, the RADIUS Auth action encountered an error.
The log message includes the user name and error message, along with the source code function name.
Refer to the <ErrorMessage> text, which contains information about the cause of the error.
<Session ID> LDAP module: WARNING: <Action> <Object> failed in <FunctionName>(): <ErrorMessage> (ErrorCode)
Specifies that the LDAP Auth or Query action encountered an error during access policy processing.
Action has one of the values:
- query with
- authentication with
Object has one of the values:
- Filter
- <AdminUserName>
- <UserName>
The message also includes the error message and the source code function name.
Refer to the <ErrorMessage> text, which contains information about the cause of the error.
<Session ID> LDAP module: ERROR: <Action> <Object> failed in <FunctionName>(): <ErrorMessage> (ErrorCode)
Specifies that the LDAP Auth or Query action encountered an error during access policy processing.
Action has one of the values:
- query with
- authentication with
Object has one of the values:
- Filter
- <AdminUserName>
- <UserName>
The message also includes the error message and the source code function name.
Refer to the <ErrorMessage> text, which contains information about the cause of the error.
<Session ID> EndPoint inspection data is not valid: Agent Result: %s SessionID: %s DeviceInfo: %s Token: %s Signature: %s
Specifies that an error occurred while reading the received request from the remote client during access policy processing.
The received request has invalid end-point inspection data. The log message logs various parts of the inspection data.
No troubleshooting information available.
<Session ID> %s is %s
Specifies the session variable name and its corresponding value.
<Session ID> process_request(): ERROR: Profile '%s' was not found
Specifies that an error occurred while the system was reading the received request from a remote client during access policy processing.
The request received is for a profile that does not exist.
This can happen if the access profile has been deleted while the remote client is processing the access policy.
No troubleshooting information available.
Access to invalid URI: (URI=<URI String>)
She system did not recognize a URI request.
Attempt to access renderer externally: (URI=<URI String>)
Indicates that a client directly accessed one or more resources inside the renderer directory. This is a security violation and the system does not allow it. The system logs the corresponding URI here.
An attempt by a client to access a resource on the internal HTTP daemon or service has been detected by the system. If the user request is associated with a session ID, you can determining the client IP address from the log messages.
Invalid Session ID <Client Session ID> Expect (<Session ID>) (URI=<URI String>)
The incoming request did not correspond to any known session ID in the system. The corresponding URI is also logged.
Invalid Client IP: we have=<IP Address> client ip=<Client IP Address> (URI=<URI String>)
The client IP of the incoming request did not match that stored internally for this session.
Attempt to access protected resource w/o valid session (URI=<URI String>)
This log message indicates that the system received a request for a protected resource from a client with an empty session ID.
Request to a protected resource w/o session ID (<URI String>)
A request to a protected resource was received with an empty session ID.
User Agent: <User Agent Name>
Specifies that the system ran out of licenses while processing user session requests. All available licenses are already in use.
CCA: Found a valid cert - adding it to the MEMCACHED
Specifies that a valid client certificate is received from remote client. The client certificate is stored in the session database.
Client cert result = <Result Status>
The result of the failed client cert authentication: revoked, unable to verify or another result.
Client Cert Auth using OCSP: Status code = <Auth Status>
Logs the result of OCSP authentication.
0 : Success
1 : Failure
-1: Error
2 : Not authenticated
Check the OCSP Responder and OCSP profile configuration settings. The reason for the failure will be listed in the access control log file.
Client Cert Auth using CRLDP: Status code = <Auth Status>
Logs the result of Client Cert Authentication using CRLDP.
0 : Success
1 : Failure
-1: Error
2 : Not authenticated
Check the CRLDP server and CRLDP profile configuration settings. The reason for the failure will be available in the access control log file.
Specifies that the client certificate the system received from the remote client has been revoked.
Specifies that the client certificate the system received from the remote client is not a valid PKI certificate.
Specifies that the client certificate the system received from the remote client could not be authenticated using OCSP. An error occurred during authentication.
Check the OCSP Responder and OCSP profile configuration settings. The reason for the failure will be available in the access control log file.
Specifies that the client certificate the system received from the remote client could not be authenticated using OCSP. An error occurred during authentication.
Check the OCSP Responder and OCSP profile configuration settings. The reason for the failure will be available in the access control log file.
Client SSL encryption: <Cipher Version> (<Cipher Name>,<Cipher Bits Size>)
Logs the SSL cipher information for the SSL session with the remote client.
Specifies that a valid client certificate was received from the remote client. Logs the Common Name (CN) field from the received certificate.
Specifies that an error occurred during user session processing and the user is being redirected to an error page. This page is shown to the user, and the user session is removed. The error code points to one of the customizable error messages.
All session variables and the session are removed from memory.
A request for the logout page was received, and the user was redirected to the logout page.
Failed to allocate client IP address for session (<Session ID>)
There is no client IP address assigned for the network access resource for this session.
Value from the session.assigned.clientip session variable is assigned to the client IP address. Either the session variable does not exist or the Session DB failed to read the variable value.
failover_id <Failover ID>
Each UNIT has a unique failover_id similar to the Unit ID used in High Availability.
Setting unit id <Failover ID> as part of session
Each UNIT has a unique failover_id similar to the Unit ID. This is used for High Availability.
Session data was deleted when failover occurred. The session is from the other UNIT and was in the middle of the access policy process.
Table E.2 lists common Kerberos error messages that you may encounter.
Pre-authentication failed while getting initial credentials
Client credentials have been revoked while getting initial credentials
Client not found in Kerberos database while getting initial credentials
Password incorrect while getting initial credentials
Password change rejected. Please try again.
A new password is rejected by the Active Directory server. For example, the current password may have been entered as the new password, or the password length is too short.
Table of Contents   |   << Previous Chapter

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)