Applies To:

Show Versions Show Versions

Manual Chapter: Configuring Anomaly Detection
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Anomaly detection is a way of detecting patterns in traffic that do not conform with normal behavior, such as an increase in latency or the transactions per second rate. Application Security Manager provides ways for you to configure the system to detect and mitigate anomalies, including:
Denial-of-service (DoS) attacks: Detects the spikes and anomalies in Layer 7 (application layer) traffic.
Brute force attacks: Protects against hackers forceful attempts to gain access to a web site.
IP address based attacks: Prevents attacks originating from a specific IP address.
Bot detection and web scraping: Prevents automated extraction of data from web sites.
A denial-of-service attack (DoS attack) is an attempt to make a computer resource unavailable to its intended users. The DoS attack generally consists of the concerted, malevolent efforts to prevent an Internet site or service from functioning efficiently or at all, temporarily or indefinitely. Perpetrators of DoS attacks typically target sites or services, such as banks, credit card payment gateways, and e-commerce web sites.
One common method of attack involves saturating the target (victim) machine with external communications requests, so that the target system cannot respond to legitimate traffic, or responds so slowly as to be rendered effectively unavailable. In general terms, DoS attacks are implemented by forcing the targeted computer to reset, or by consuming its resources so that it can no longer provide its intended service, or by obstructing the communication media between the intended users and the victim so that they can no longer communicate adequately.
Denial-of-service attacks are also known as HTTP-GET attacks or page flood attacks. The attacks can either be initiated from a single user (single IP address) or from thousands of computers (distributed DoS attack), which overwhelms the target system. A page flood attack (or HTTP flood attack) may resemble the patterns of normal Web surfing, making it harder for automated tools to differentiate between legitimate Web traffic and an attempted attack.
Application Security Manager considers traffic to be a DoS attack based on calculations for transaction rate (TPS-based) or latency (latency-based). You can configure which calculations you want the system to use.
Transaction rate detection interval
The average number of requests per second sent for a specific URL, or by a specific IP address. The system calculates this number every minute.
Transaction rate history interval
The average number of requests per second sent for a specific URL, or by a specific IP address. The system calculates this number every hour.
If the ratio of the transaction rate detection interval to the transaction rate history interval is greater than the specific percentage you configure on the DoS Attack Prevention screen (the TPS increased by percentage), the system considers the URL to be under attack, or the IP address to be suspicious. To prevent further attacks, the system drops requests for this URL, and drops requests from the suspicious IP address.
Latency detection interval
The average time it takes for the system to respond to a request for a specific URL, for each web application, over the last minute. This average is updated every second.
Latency history interval
The average time it takes for the system to respond to a request for a specific URL, for each web application, over the last hour. This average is updated every minute.
If the ratio of the latency detection interval to the latency history interval is greater than the percentage you configure on the DoS Attack Prevention screen (the Latency increased by percentage), the system detects that this URL is under attack.
You can configure the Application Security Manager to monitor latency of Layer 7 (application layer) traffic and transactions per second to detect the spikes and anomalies in the typical traffic pattern. The spikes and anomalies may indicate an attempted attack. The system rejects suspicious traffic based on absolute IP addresses and your configuration.
1.
In the Application Security navigation pane, expand Application Security and click Anomaly Detection.
The DoS Attack Prevention screen opens.
3.
For Operation Mode, select how you want the system to react to DoS attacks:
Off
Do not check for DoS attacks. This is the default setting.
Transparent
Log reporting data when the system detects a DoS attack.
Blocking
Drop connections coming from an attacking IP address or going to a URL being attacked.
4.
For the Detection Mode, select the way you want the system to look for DoS attacks:
TPS-based
Determines DoS attacks from the client side based on the number of requests per second sent to a specific URL, or the number of transactions per second coming from a specific IP address. This is the default setting.
Latency-based
Determines DoS attacks from the server side based on the average time it takes for the system to respond to a request for a specific URL.
5.
If you select Latency-based, specify the threshold values for Suspicious Criteria:
Latency increased by: Specifies that the system considers traffic to be an attack, if the latency has increased by this percentage. The default value is 500%.
Latency reached: Specifies that the system considers traffic to be an attack, if the latency is equal to or greater than this value. This setting provides an absolute value, so, for example, if an attack increases latency gradually, the increase might not exceed the Latency Increased by threshold and would not be detected. If server latency reaches the Latency reached value, the system considers traffic to be an attack even if it did not meet the Latency increased by criterion. The default value is 10000 ms.
Minimum Latency Threshold for detection: Specifies that the system considers traffic to be an attack if the detection interval for a specific URL equals, or is greater than, this number, and at least one of the Latency increased by or Latency reached numbers was reached. If the detection interval is lower than this number, the system does not consider this traffic to be an attack even if one of the Latency increased by or Latency reached numbers was reached. The default setting is 200 ms.
6.
For the Prevention Policy setting, select one or more options to determine how you want the system to handle a DoS attack:
Source IP-Based Client-Side Integrity Defense
Checks whether clients are legitimate by sending a challenge to suspicious IP addresses and waiting for a proper response. Legal browsers can respond, whereas illegal scripts cannot. The default is disabled.
URL-Based Client-Side Integrity Defense
Checks whether clients are legitimate by sending a challenge to suspicious URLs and waiting for a proper response. Legal browsers can respond, whereas illegal scripts cannot. This setting enforces strong protection and prevents distributed DoS attacks but affects more clients. The default is disabled.
Source IP-Based Rate Limiting
Check to drop requests from suspicious IP addresses. Application Security Manager drops connections to limit the rate of requests to the average rate prior to the attack, or lower than the absolute threshold specified by the IP detection TPS reached setting. The default is enabled.
URL-Based Rate Limiting
Check to indicate that when the system detects a URL under attack, Application Security Manager drops connections to limit the rate of requests to the URL to the average rate prior to the attack.
7.
For IP Detection Criteria, type the threshold values:
Note: This setting appears only if Prevention Policy is set to Source IP-Based Client Side Integrity Defense and/or Source IP-Based Rate Limiting.
TPS increased by: Specifies that the system considers an IP address to be that of an attacker, if the transactions (requests) sent per second have increased by this percentage. The default value is 500%.
TPS reached: Specifies that the system considers an IP address to be suspicious if the number of transactions (requests) sent per second from an IP address is equal to or greater than this value. This setting provides an absolute value, so, for example, if an attack increases the number of transactions gradually, the increase might not exceed the TPS increased by threshold and would not be detected. If the TPS reaches the TPS reached value, the system considers traffic to be an attack even if it did not meet the TPS increased by criterion. The default value is 200 TPS.
If either of these criteria is met, the system handles the attack according to the Prevention Policy settings.
8.
For URL Detection Criteria, type the threshold values:
Note: This setting appears only if Prevention Policy is set to URL-Based Client Side Integrity Defense and/or URL-Based Rate Limiting.
TPS increased by: Specifies that the system considers a URL to be attacked, if the number of transactions (requests) sent per second to the URL have increased by this percentage. The default value is 500%.
TPS reached: Specifies that the system considers a URL to be suspicious if the number of transactions (requests) sent per second to the URL is equal to or greater than this value. This setting provides an absolute value, so, for example, if an attack increases the number of transactions gradually, the increase might not exceed the TPS Increased by threshold and would not be detected. If the TPS reaches the TPS reached value, the system considers traffic to be an attack even if it did not meet the TPS increased by criterion. The default value is 1000 TPS.
If either of these criteria is met, the system handles the attack according to the Prevention Policy settings.
9.
For the Prevention Duration setting, specify the length of time for which the system mitigates DoS attacks:
Unlimited: Click if you want the system to perform attack prevention until it detects the end of the attack.
Maximum: Click and type a value, in seconds. The system prevents detected DoS attacks for the time configured here (even if the attack is still occurring), or until the system detects the end of the attack, whichever is sooner.
10.
In IP Address Whitelist, add the IP addresses or subnets that do not need to be checked for DoS attacks.
11.
Click Save to save the detection and prevention criteria.
12.
To put the security policy changes into effect immediately, click the Apply Policy button in the editing context area.
You can view details about DoS attacks that the system detected and logged. For information about the DoS Attacks reports, refer to Viewing DoS Attacks reports. You can also configure remote logging support for DoS attacks when creating a logging profile. For information about creating remote logging profiles, refer to Configuring a logging profile for remote storage.
You can configure the Application Security Manager to detect, report on, and prevent Layer 7 brute force attack attempts,. Brute force attacks are attempts to break in to secured areas of a web application by trying exhaustive, systematic user name/password combinations to discover legitimate authentication credentials. Malicious users send high volumes of these combinations, often scripted, to avoid security mechanisms. In this automated scenario, one malicious user can make thousands of login attempts per minute.
In Application Security Manager, you can configure the login URL of a web application to protect, the mitigation methods, and the access validation criteria for logon responses. With brute force protection, the system can monitor traffic to detect excessive failures to authenticate, monitor suspicious IP addresses (generating a high volume of login attempts), and detect other anomalies in the typical traffic pattern for the login URL.
History interval: Failed login attempts for the past hour (updated every minute).
Detection interval: Failed login attempts for the past minute (updated every second).
The system considers it to be a brute force attack if either the detection interval exceeds the history interval by a percentage (500% by default), or if the failed login attempts rate is reached (1000 per second by default).
1.
In the Application Security navigation pane, expand Application Security, point to Anomaly Detection, and click Brute Force Attack Prevention.
The Brute Force Attack Prevention screen opens.
3.
Click the Create button.
The New Brute Force Protection Configuration screen opens. Next, you configure the login information.
1.
For Login URL:
Select Explicit (to specify one URL) or Wildcard (to specify a pattern that will match multiple URLs).
The system automatically adds any login URL (explicit or wildcard) that you specify on this screen to the security policys list of URLS.
2.
For Authentication Type, select the method by which the application server validates user logon credentials:
HTML Form
Specifies that the code of the login URL is in HTML format. This is the default setting.
HTTP Basic Authentication
Specifies that the web server uses HTTP basic authentication to authenticate users.
HTTP Digest Authentication
Specifies that the web server uses HTTP digest authentication to authenticate users.
NTLM
Specifies that the web server uses NT LAN Manager (NTLM) authentication to authenticate users.
3.
For Username Parameter Name, type the user name parameter included in the code of the HTML form. When the system detects this parameter with the password parameter, the system recognizes the request as a login attempt. (Applies only to the HTML Form authentication type.)
4.
For the Password Parameter Name setting, type the password parameter written in the code of the HTML form. When the system detects this parameter with the username parameter, the system recognizes that request as a login attempt. (Applies only to the HTML Form authentication type.)
Next, you can configure session-based mitigation.
Session-based mitigation counts the number of failed login attempts that occur during one session for an IP address. When the system detects a brute force attack, it triggers the Maximum login attempts are exceeded violation, and applies the blocking policy.
To configure session-based mitigation, use the settings in the Session-based Brute Force Protection area of the New Brute Force Protection Configuration screen.
1.
Above the Session-based Brute Force Protection area, click the Blocking Settings link to open the Blocking Policy screen where you can configure the blocking actions taken when detecting excessive login attempts.
Note: See Configuring the blocking policy, for more information.
2.
For Login Attempts From The Same Client, type the number of times a user can attempt to log on before the system blocks the request. The default value is 5.
3.
For Re-enable Login After, type the number of seconds the user must wait to log on after they have been blocked.
Next, you can configure dynamic brute force protection.
Dynamic mitigation detects and mitigates brute force attacks based on statistical analysis of the traffic. You configure dynamic mitigation to determine when the system should consider the logon URL to be under attack, and how to react to an attack. The system mitigates attacks when the volume of unsuccessful login attempts is significantly greater than the typical number of failed logins.
To configure dynamic brute force protection, use the settings in the Dynamic Brute Force Protection area of the New Brute Force Protection Configuration screen.
1.
For Operation Mode, select how to handle brute force attacks:
Off
Do not monitor traffic to detect brute force attacks.
Transparent
Issue reporting data only on attacks. Do not reject illegal requests. This is the default setting.
Blocking
Reject illegal requests and log reporting data.
2.
For the Detection Criteria setting, specify when to consider login attempts an attack.
Failed Logins Attempts increased by
For all IP addresses tracked, it is an attack if the number of failed login attempts has increased by this percentage over the normal number of failed logins, and the Minimum Failed Login Attempts is reached.
Failed Login Attempts Rate reached
For all IP addresses tracked, it is an attack if the logon rate reaches this number, and the Minimum Failed Login Attempts is reached.
Minimum Failed Login Attempts
For all IP addresses tracked, it is an attack if the number of login attempts is greater than or equal to this number, and either of the two other settings is reached. This setting prevents attack mitigation on typical traffic volumes.
3.
For the Prevention Policy setting, select the methods you want the system to use to mitigate an attack (the methods are applied in the order listed).
Source IP-Based Client-Side Integrity Defense
Check to determine whether the client is a legal browser or an illegal script by sending a challenge to the suspicious IP address and waiting for a proper response. Legal browsers can respond, whereas illegal scripts cannot. The default is disabled.
URL-Based Client-Side Integrity Defense
Check to determine whether the client is a legal browser or an illegal script by sending a challenge to the IP addresses trying to access the login URL and waiting for a proper response. Legal URLs can respond, whereas illegal scripts cannot. The default is disabled.
Source IP-Based Rate Limiting
Check to drop requests from suspicious IP addresses. Application Security Manager drops connections to limit the rate of requests to the average rate prior to the attack. The default is enabled.
URL-Based Rate Limiting
Check to indicate that when the system detects a URL under attack, Application Security Manager performs rate limiting and limits the rate of all logon requests to the normal level. The default is enabled.
4.
For Suspicious Criteria (per IP address), specify how to identify traffic that may be an attack. If at least one of the criteria is met, the system treats the IP address as an attacker, and prevents the attacker from trying to guess the password. The system also limits the number of login attempts to the normal level.
Failed Login Attempts increased by
Type a number. Login attempts from an individual IP address are considered an attack if the number of failed login attempts has increased by this percentage over the normal number of failed logins. The default setting is 500 percent.
Failed Login Attempts Rate reached
Type a number. An individual IP address is suspicious if the number of login attempts per second from an IP address is equal to or greater than this number. The default setting is 1000 login attempts per second.
5.
For the Prevention Duration setting, specify the length of time for which the system mitigates brute force attacks.
Unlimited
Specifies that after the system detects and mitigates a brute force attack, it performs attack prevention until it detects the end of the attack.
Maximum
Specifies that after the system detects and mitigates a brute force attack, it performs attack prevention either for the time configured here (even if the system detects that the attack continues), or until the system detects the end of the attack, whichever is earlier. Type a value, in seconds, in the box.
6.
In IP Address Whitelist, add the IP addresses or subnets that do not need to be checked for brute force attacks.
Next, you can define validation criteria to apply to the response of the login URL.
For Access Validation, define at least one validation criterion for responses of the login URL:
A string that should appear in the response
Type a string that must appear in the response for the system to detect a successful login attempt; for example, Successful Login.
A string that should NOT appear in the response
Type a string that indicates a failed login attempt; for example, Authentication failed.
Expected HTTP response status code
Type an HTTP response code that is sent when the user successfully logs in; for example, 200.
Expected validation header name and value (for example, Location header)
Type a header name and value that is sent when the user successfully logs in.
Expected validation domain cookie name
Type a defined domain cookie name that is sent when the user successfully logs in.
Expected parameter name (added to URI links in the response)
Type a parameter that is sent when the user successfully logs in.
Next, you can finish brute force protection configuration.
1.
At the bottom of the New Brute Force Protection Configuration screen, click Create.
The screen refreshes, and you see your protected login URL in the list.
2.
To put the security policy changes into effect immediately, click the Apply Policy button in the editing context area.
You can configure the IP Enforcer to perform enforcement based on IP address. In blocking mode, Application Security Manager drops requests for any IP address if more than the maximum number of blocked violations occur in one minute. The system detects an attacker and drops all requests from that IP address for a specified time interval. The IP Enforcer can also operate in a transparent mode where it reports IP addresses that exceed the blocked violation threshold but it does not drop the requests.
Note: IP address enforcement stops traffic only if the security policys enforcement mode is Blocking, and violations must have the Block flag enabled (on the Blocking Policy screen).
When the IP Enforcer is configured, you can view IP Enforcer statistics to investigate the prevented IP addresses, dropped requests from that IP address, and violations that occurred for each IP address. For more information, see Viewing IP Enforcer statistics.
1.
In the Application Security navigation pane, point to Anomaly Detection and click IP Enforcer.
The IP Enforcer Configuration screen opens.
3.
In the IP Enforcer Configuration area, for the Operation Mode setting, select the mode for IP enforcement:
Off
Do not perform IP-based enforcement. This is the default setting.
Transparent
Issue reporting data. In this mode, if the violation threshold is exceeded, the system logs the attack data, but does not prevent access to the application by the offending client.
Blocking
Prohibit access by this IP address if more than the maximum number of blocked violations occur in one minute.
4.
For the Violation Threshold setting, type the maximum number of blocked violations that you want to allow per minute for each IP address. The default value is 10.
5.
For the Prevention Duration setting, type the number of seconds you want to block (or report) requests. The default value is 60
6.
Click Save to save the IP enforcement configuration.
7.
To put the security policy changes into effect immediately, click the Apply Policy button in the editing context area.
You can configure Application Security Manager to detect and prevent various bot activities and web scraping on web sites that it is protecting. Web scraping is a technique for extracting information from web sites typically using automated programs, or bots (short for web robots).
Dropped request
If the system cannot check requests for human activity, the request is dropped with no further checking. This action occurs only if the Web scraping detected violation is set to Block, and does not occur if the security policy is running in transparent mode or if the violation is set to Learn or Alarm only.
Grace interval
The grace interval is how many requests the system reviews while trying to detect whether the client is human. During the grace interval, requests are not blocked or reported. What occurs next depends on whether or not the system detects human activity:
If the system detects human activity
The grace interval ends and the system handles the number of requests specified in the Safe Interval, then restarts the grace interval and starts checking again.
If the system does not detect human activity
The system issues the Web Scraping Detected violation until it reaches the number of requests in the Unsafe Interval. If the violation is set to Block, the system blocks requests during this time. In transparent mode or if the violation is set to Alarm only, the violation is logged and requests are permitted. After reaching the Unsafe Interval, the system restarts the grace interval and starts checking again.
If your environment uses legitimate automated tests, you can create a white list of IP addresses or address ranges from which to allow access. The system does not perform web scraping detection on these addresses. In addition, the system allows requests from well-known search engines and legal web robots including the following:
Google (googlebot.com)
Yahoo (crawl.yahoo.net)
MSN (search.msn.com)
ask.com (ask.com)
AOL (using googlebot.com)
1.
In the Application Security navigation pane, expand Application Security, point to Anomaly Detection, then click Web Scraping.
The Web Scraping screen opens.
3.
For Grace Interval, type the number of requests to allow while determining whether the client is human. The default value is 100.
4.
For Unsafe Interval, type the number of requests that cause the Web Scraping Detected violation if no human activity is detected during the grace period. When the number is reached, reactivate the grace period. The default value is 100.
5.
For Safe Interval, type the number of requests to allow after human activity is detected and before reactivating the grace threshold to check again for non-human clients. The default value is 2000.
6.
In IP Address Whitelist, add the IP addresses or subnets that do not need to be checked for web scraping.
7.
Click Save.
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)