Applies To:

Show Versions Show Versions

Manual Chapter: Refining the Security Policy Using Learning
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

12 
You can use learning process resources to help if you are building a security policy manually. When you send client traffic through the Application Security Manager, the learning data provides information on requests or responses that do not comply with the current security policy and triggered a violation. The reason for triggering a violation can be either a false positive (typically seen during the process of building a policy), or an actual attack on the site.
The system generates learning suggestions for requests that cause violations and do not pass the security policy checks. You examine the requests that cause learning suggestions, and then use the suggestions to refine the security policy. In some cases, learning suggestions may contain recommendations to relax the security policy due to attacks. When dealing with learning suggestions, make sure to relax the policy only where false positives occurred, and not in cases where a real attack caused a violation.
An internal system process that examines the security policy violations that the system identifies, and generates learning suggestions, or ways to update the security policy, based on those policy violations. As visitors move through the web application, the Learning Manager captures requests that contravene the current security policy settings, and records the learning suggestions on the Traffic Learning screen.
A screen that displays learning suggestions that the Learning Manager generates. The learning suggestions are categorized by violation type, and can represent actual threats or false-positives. Learning suggestions are for the currently active security policy. When you accept a learning suggestion, you are updating the currently active security policy.
A screen that summarizes the security policy entities in staging or with tightening enabled, that may have learning suggestions, and may be ready to be enforced. For file types, parameters, URLs, cookies, and signatures, you can review the entities, and decide whether to add them to the security policy.
A screen that lists the file types, URLs, and flows that you have instructed the Learning Manager to disregard, that is, to stop generating learning suggestions for. Typically, the ignored entities are items that you do not want to be a part of the security policy.
A screen that lists IP address exceptions with specific characteristics that you can configure. You can instruct the system not to generate learning suggestions for traffic sent from any of these IP addresses.
View Full Request Information screen
A screen that lists any violations and details associated with a request. You can review this information, and then if you want to accept the learning suggestion, click the Learn button to update the active security policy. To display the View Full Request Information screen, from the Reporting Requests screen, click a Requested URL in the Requests List.
If you are generating a security policy automatically, the system handles all learning for you, adjusting the security policy based on traffic characteristics. In that case, the learning screens show only the elements it is in the process of learning.
The Learning Manager generates learning suggestions when the Learn flag is enabled for the violations on the Policy Blocking Settings screen. (See Configuring the blocking actions, for how to set the flag.) When the system receives a request that triggers a violation, the Learning Manager then updates the Traffic Learning screen with learning suggestions based on the violating request information (see Figure 12.1 for an example screen). From this screen, you can review the learning suggestions to determine whether the request triggered a legitimate security policy violation, or if the violation represents a need to update the security policy.
Making decisions about which learning suggestions to use requires some general understanding of application security, and specific knowledge of the protected application (for example, recognizing valid traffic). Often, you should consider accepting a learning suggestion when you see that it has occurred multiple times, from many different source IP addresses. Repeated learning suggestions typically indicate valid traffic behavior that warrants relaxing the security policy.
The Traffic Learning screen also displays violations for which the system does not generate learning suggestions. Typically, these violations are related to RFC compliance and system resources; the resolution for these violations may be to disable the violation or subviolation rather than to perform any specific configuration. The system displays these violations along with the learning suggestions to ease the security policy management tasks.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning area, click a violation hyperlink to view the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
The system displays the learning suggestion details or a list of requests.
Note: In learning suggestions, the Application Security Manager displays and processes non-printable characters, that is, control characters, in the same manner as it displays and processes other characters. For example, the system displays the space character as 0x20.
For example, if the security policy contains a wildcard global parameter and the system detects a request for an explicit parameter whose name contains an illegal meta character:
Using real traffic, the system suggests adding the parameter to the security policy with the parameter level detected in the request. So, the system might suggest adding both a URL parameter and the URL in which the parameter was sent.
Using existing entities, the system suggests adding the parameter to the security policy as a global parameter if the wildcard parameter was defined as a global parameter.
1.
On the Main tab, expand Application Security, point to Policy Building, Manual, then click Advanced Settings.
The Advanced Settings screen opens.
Learn suggestions for parameter violations based on traffic: Specifies that system presents learning suggestions based on real traffic, and is not limited to the current properties of entities that exist in the security policy.
Learn suggestions for parameter violations based on existing security policy parameter settings: Specifies that system offers learning suggestions based entities in the security policy. This is how the system behaves in versions prior to version 11.2.0.
Note: This setting is only applicable for the following violations: Attack signature detected, Illegal meta character in value, and Illegal meta character in parameter name.
3.
Click Save.
Before you process a learning suggestion, it is very helpful to examine the details of the request that caused the learning suggestion. First, click the name of the violation, and then click either the occurrences or the request itself, according to what is displayed on the screen.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning area, click a violation hyperlink to view either the Requests List, or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
4.
In the Occurrences column, click the number.
The Requests List popup screen opens, and displays all of the requests that triggered the learning suggestion.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning section, click a violation hyperlink to view either the request or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
The system displays the request or request elements that caused the learning suggestions for the selected violation.
4.
In the Occurrences column, if available, click the number.
The Requests List popup screen opens, and displays all of the requests that contained an item that triggered the learning suggestion.
Note: Some violations have no Occurrences number.
5.
In the Recent Incidents column (if attack signatures were detected), click the number.
The Requests List popup screen opens, and displays all of the requests that contained an item that triggered the learning suggestion.
6.
In the Requests List area of the popup screen, in the URL column, click a URL link.
The View Request Information screen opens in the popup screen, where you can review the request that triggered the learning suggestion.
7.
For each violation with a Learn button, click Learn to go back to the violation learning screen where you can accept or clear the learning suggestions for the security policy one value at a time.
8.
To view the actual contents of the request, click Full Request. and when you are done looking at the request details, click Close.
9.
On the screen showing learning suggestions for the violation, to accept the suggestion and change the security policy, click Accept.
10.
To remove learning suggestions without changing the security policy, select the ones to remove, and then click the Clear button.
If you want to review requests for a security policy that triggers learning suggestions, you can do so on the Requests screen.
1.
On the Main tab, expand Application Security and click Reporting.
The Requests screen opens.
2.
3.
For the Security Policy setting, select the name of the security policy for which you want to see requests.
4.
From the Request Type list, select All.
5.
Click the Go button.
The screen refreshes, and in the Requests List area, you see the requests for the selected security policy. Note that you only see staging suggestions if the logging profile for the security policy is set to log all requests.
Application Security Manager generates learning suggestions throughout the life of the security policy. When the system detects violations of a security policy, the violations may be related to a real attack, and may therefore warrant more careful inspection before being accepted into the security policy.
You can review learning suggestions (violations) on the Traffic Learning screen, and accept or clear each suggestion, as described following. You can also view learning suggestions from the Staging-Tightening Summary screen, as described in Working with entities in staging or with tightening enabled.
Note: When using automatic policy building to build a security policy, Policy Builder handles most learning suggestions by adjusting the policy. It is possible to see suggestions on the Traffic Learning screen even after the security policy is stable. You can review the suggestions and accept any that are caused by false positives.
The system provides learning suggestions for many of the violations. By default, learning suggestions are presented for the active policy. When you accept a learning suggestion, the system updates the current edited security policy to accept the request entity that triggered the violation.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
3.
Click a violation hyperlink.
The learning suggestions properties screen opens. Note that the screens vary for different violations.
4.
Select one or more learning suggestions, and then click the Accept, Apply, or Allow button, depending on the violation.
The system updates the security policy with the element in the request that caused the learning suggestion.
When you clear a learning suggestion, the system deletes the learning suggestion, and does not update the security policy. The system continues to generate learning suggestions for future instances of the violation.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
a)
Select one or more violations, and then click Clear.
A confirmation popup appears.
b)
Click OK.
The system deletes all of the learning suggestions and removes the violation from the list without changing the security policy.
a)
Click a violation hyperlink.
The violation properties screen opens.
b)
Select one or more learning suggestions, and then click Clear.
A confirm delete popup screen opens.
c)
Click OK.
The system deletes the learning suggestion without changing the security policy.
Note: For a description of the violation types, go to the Policy Blocking Settings screen and click the next to the violation name. You can also refer to Appendix A, Security Policy Violations.
You use the Staging-Tightening summary (shown in Figure 12.2) to review file types, URLs, parameters, cookies, and signatures that are in staging or with tightening enabled, and you can delve into the details to see if you want to add or update these entities in the security policy. You can add selected entities to the security policy, or you can enforce all of the entities that are ready to be enforced.
You can click the numbers in the columns to display details about the entities that are in staging or with tightening enabled. For example, Figure 12.3 shows the learning suggestions that are displayed when you click the number link in the Have Suggestions column of File Types.
When you review the learning suggestions, you can clear them or go back to the staging-tightening summary and enforce the entities. You can also click a learning suggestion in the list to have the security policy learn it, as described in Accepting a learning suggestion.
You can perform tightening on wildcard entities (file types, URLs, parameters, and cookies) to learn explicit entities. When you enable tightening for a wildcard entity, and the system receives a request that contains an entity that matches the wildcard entity, the system generates a learning suggestion for the found entity. You can then review the new entities, and decide which are legitimate entities for the web application.
Tightening allows you to develop a more specific policy that is more accurate and in alignment with the traffic. Such a policy can provide better security, but requires more tuning to make sure all the specific entities that you add are accurately configured.
Tip: Use tightening on wildcard entities to build the security policy with explicit entities of this type. For additional information on wildcard entities, see Chapter 8, Working with Wildcard Entities.
You can perform staging on file types, URLs, parameters, enforced cookies, and signatures to learn properties of entities, such as:
For URLs, learn meta characters (wildcard URLs only) and illegal content type violations including those associated with XML and JSON payloads
When an entity is in staging, the system does not block any requests for this entity. Instead, it posts learning suggestions for staged entities in the Violations Found for Staged Entities table in the request details.
Tip: Use staging on wildcard entities to build the security policy without specifying explicit entities of this type.
Staging is also useful when a site update occurs for a web application. Without staging, you might have to change the blocking policy enforcement mode to transparent for the entire web site to discover any new URLs or parameters in the updated web application. With staging, you can add any new URLs or parameters to the security policy, and place only the new entities in staging allowing the system to generate learning alerts.
If a file type, URL, parameter, or cookie is in staging or has tightening enabled, the system displays a status icon in the Staging or Tightening column of the file types, URLs, parameters, or cookies. For example, Figure 12.4 shows the Allowed File Types List with one files type in staging, and the * wildcard with tightening enabled.
The icons in the Staging and Tightening columns provide details about the status of the file type, URL, or parameter. Move the cursor over the icon to see when the entity was placed in staging and the last time the properties of this entity were changed (the Last staging/tightening event time date and time).
On the Attack Signatures List screen, you can view the status of attack signatures that are in staging, as shown in Figure 12.5.
If the signature is in staging, the Learn column displays whether the signatures is in staging and for how long. For more information about attack signature staging, refer to Understanding attack signature staging.
After you create a security policy and traffic is sent to the web application, new entities are added by means of tightening, and existing entities are modified through staging. You can review the entities that are in staging or with tightening enabled and add the entities to the security policy. When the staging or tightening period is over and no learning suggestions are added for the staging period duration (the default is 7 days), the file type, URL, parameter, cookie, or signature is considered ready to be enforced. You can enforce the entities one at a time.
1.
On the Main tab, expand Application Security, point to Policy Building, Manual and click Staging-Tightening Summary.
The Staging-Tightening Summary screen opens.
3.
In the Staging-Tightening Summary, check to see if a number appears in the In Staging-Tightening column.
A number greater than zero indicates that entities of that type are in staging or with tightening enabled.
4.
Click the number in the In Staging-Tightening column.
The allowed file types, URLs, parameters, cookies, or signatures list opens showing the entities that you can enforce.
6.
Click Enforce.
The system takes the following actions:
To enforce all entities that are ready to be enforced
1.
On the Main tab, expand Application Security, point to Policy Building, Manual and click Staging-Tightening Summary.
The Staging-Tightening Summary screen opens.
4.
Click the Enforce Ready button.
The system takes the following actions:
For these violations, F5 Networks recommends that you review the violations, and determine whether they represent legitimate violations or false-positives. You can disable these violations if they are not applicable to your web application. Disabling a violation turns off the blocking policy so that you are no longer notified of requests that trigger the violation. Alternately, you can clear the learning suggestions, and Application Security Manager continues to issue learning suggestions for the requests.
Note: Application Security Manager does not generate learning suggestions for requests that result in the web server returning HTTP responses with 400 or 404 status codes.
If you do not want the system to display the violations that require user interpretation, you can disable the violation. The Disable Violation button disables all flags on the selected violation. The system then ignores future instances of the violation, and passes the requests on to the web application resources. Be sure that you understand the ramifications of disabling a violation before doing it.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
4.
Click the Disable Violation button.
A confirmation popup screen opens.
5.
Click OK.
The screen refreshes, and you no longer see the violation in the Traffic Learning area.
Tip: You can navigate to the Policy Blocking Settings screen to see that all flags on the selected violation are unchecked.
6.
To put the security policy changes into effect immediately, click the Apply Policy button in the editing context area.
A confirmation popup screen opens.
7.
Click OK.
The system applies the updated security policy.
When you clear a violation, the system deletes the violation, but does not update the security policy. The system continues to generate alarms for future instances of the violation, and the Learning Manager continues to generate learning suggestions relative to the violation.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
3.
In the violations list, select the box next to a violation, and then click Clear.
A Confirm Delete popup screen opens.
4.
Click OK.
The system deletes the learning suggestion.
When you reject a learning suggestion for a URL, a file type, or a flow, the Application Security Manager adds the item to the ignored entities list. When the system receives subsequent requests for those items, the system no longer generates learning suggestions for them. The system does, however, continue to log the requests.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review ignored entities.
3.
From the Manual menu, choose Ignored Entities.
The Ignored Entities screen opens showing the number of ignored entities for file types, URLs, and parameters. If ignored entities exist for an entity type, that type becomes a link that you can click to view a list of all entities logged within that category.
If you want the system to start generating learning suggestions for items that were previously added to the ignored entities list, you can remove those items from the list.
1.
On the Main tab, expand Application Security, point to Policy Building and click Manual.
The Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review ignored entities.
3.
From the Manual menu, choose Ignored Entities.
The Ignored Entities screen opens.
4.
Select the entity type whose ignored entities you want to remove, and click the Delete button.
The system removes all ignored items of the selected entity type from the ignored item status and resumes generating learning suggestions for this entity type.
For each security policy, you can create a centralized list of IP address exceptions, or IP addresses that the system should treat differently. You can specify that the system trusts certain IP addresses, never blocks or never logs traffic sent from these IP addresses, and ignores certain IP addresses in anomaly detection. You can also instruct the system not to generate learning suggestions for traffic sent from these IP addresses.
Creating a list of IP address exception is useful, for example, if your company performs penetration testing using manual or automatic scanners. When you add the IP address of the scanner, you can prevent the system from generating learning suggestions for traffic from the scanner, but still have the system make learning suggestions for other legitimate production traffic.
1.
On the Main tab, expand Application Security, point to IP Addresses, then click IP Address Exceptions.
The IP Address Exceptions screen opens.
2.
In the editing context area, ensure that the current edited web application is the one for which you want to add IP address exceptions.
3.
Click the Create button.
The New IP Address Exception screen opens.
5.
To instruct the system to always trust this IP address, for the Policy Builder trusted IP setting, select the Enabled check box.
If you enable this setting, the Policy Builder automatically adds the traffic data from this IP address to the security policy. The system adds this IP address to the Trusted IP Addresses list on the Policy Building > Automatic > Configuration screen.
6.
To have the system ignore this IP address when performing DoS prevention, brute force prevention, and web scraping detection, for the Ignore in Anomaly Detection setting, select the Enabled check box.
If you enable this setting, the system automatically adds this IP address to the IP Address Whitelists on the DoS, brute force, and web scraping screens.
7.
If you do not want the system to generate security policy suggestions for traffic from this IP address, for the Ignore in Learning Suggestions setting, select the Enabled check box.
8.
To prevent the system from blocking traffic from this IP address, for the Never Block this IP Address setting, select the Enabled check box.
9.
To instruct the system not to log requests from this IP address, for the Never log requests from this IP Address setting, select the Enabled check box.
If you enable this setting, the system does not log requests sent from this IP address, even if the traffic is illegal, and even if your security policy is configured to log all traffic.
10.
If you want the system to consider this IP address legitimate even if it is in the IP address intelligence database, for the Ignore IP Address Intelligence setting, select the Enabled check box.
11.
In the Description field, type a note about why this IP address is an exception.
12.
Click Create.
The system adds the IP address to the list of IP address exceptions.
1.
On the Main tab, expand Application Security, point to IP Addresses, then click IP Address Exceptions.
The IP Address Exceptions screen opens.
2.
In the editing context area, ensure that the current edited web application is the one with IP address exceptions you want to change.
3.
Select the IP address exception that you want to remove, and click the Delete button.
After you confirm, the IP address is removed from the IP address exceptions list (and also from the any other lists it was on such as the trusted IP address list and the anomaly detection whitelists.
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)