Applies To:

Show Versions Show Versions

Manual Chapter: Refining the Security Policy Using Learning
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

12 
You can use learning process resources to help if you are building a security policy manually. When you send client traffic through the Application Security Manager, the learning data provides information on requests or responses that do not comply with the current security policy and triggered a violation. The reason for triggering a violation can be either a false positive (typically seen during the process of building a policy), or an actual attack on the site.
The system generates learning suggestions for requests that cause violations and do not pass the security policy checks. You examine the requests that cause learning suggestions, and then use the suggestions to refine the security policy. In some cases, learning suggestions may contain recommendations to relax the security policy due to attacks. When dealing with learning suggestions, make sure to relax the policy only where false positives occurred, and not in cases where a real attack caused a violation.
Displays learning suggestions that the system generates. The learning suggestions are categorized by violation type, and can represent actual threats or false-positives. Learning suggestions are for the currently active security policy. When you accept a learning suggestion, you are updating the currently active security policy.
Summarizes the security policy entities in staging or with learn explicit entities enabled, that may have learning suggestions, and may be ready to be enforced. For file types, parameters, URLs, cookies, and signatures, you can review the entities, and decide whether to add them to the security policy.
Lists the file types, URLs, and flows that you have instructed the system to disregard, that is, to stop generating learning suggestions for. Typically, the ignored entities are items that you do not want to be a part of the security policy.
Lists IP address exceptions with specific characteristics that you can configure. You can instruct the system not to generate learning suggestions for traffic sent from any of these IP addresses.
View Full Request Information screen
Displays any violations and details associated with a request. You can review this information, and then if you want to accept the learning suggestion, click the Learn button to update the active security policy. To display the View Full Request Information screen, from the Event Logs: Application: Requests screen, click a Requested URL in the Requests List.
If you are generating a security policy automatically, the system handles all learning for you, adjusting the security policy based on traffic characteristics. In that case, the learning screens show only the elements it is in the process of learning.
Application Security Manager generates learning suggestions when the Learn flag is enabled for the violations on the Application Security: Blocking: Settings screen. (See Configuring the blocking actions, for how to set the flag.) When the system receives a request that triggers a violation, the system updates the Manual Traffic Learning screen with learning suggestions based on the violating request information (see Figure 12.1 for an example screen). From this screen, you can review the learning suggestions to determine whether the request triggered a legitimate security policy violation, or if the violation represents a need to update the security policy.
Making decisions about which learning suggestions to use requires some general understanding of application security, and specific knowledge of the protected application (for example, recognizing valid traffic). Often, you should consider accepting a learning suggestion when you see that it has occurred multiple times, from many different source IP addresses. Repeated learning suggestions typically indicate valid traffic behavior that warrants relaxing the security policy.
The Manual Traffic Learning screen also displays violations for which the system does not generate learning suggestions. Typically, these violations are related to RFC compliance and system resources; the resolution for these violations may be to disable the violation or sub-violation rather than to perform any specific configuration. The system displays these violations along with the learning suggestions to ease the security policy management tasks.
Note: The Manual Traffic Learning screen displays violations only when the system has detected them in a request. If no violations have occurred, the screen appears blank.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning area, click a violation hyperlink to view the specific elements in the request that triggered the security policy violation, and the corresponding learning suggestion.
The system displays the learning suggestion details or a list of requests.
Note: In learning suggestions, the Application Security Manager displays and processes non-printable characters, that is, control characters, in the same manner as it displays and processes other characters. For example, the system displays the space character as 0x20.
Explicit learning settings specify when Policy Builder adds, or suggests you add, explicit entities to the security policy. You can adjust the explicit entities learning settings for file types, URLs, and parameters in the general policy building settings as described in Configuring explicit entities learning.
Before you process a learning suggestion, it is very helpful to examine the details of the request that caused the learning suggestion. First, click the name of the violation, and then click either the occurrences or the request itself, according to what is displayed on the screen.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning area, click a violation hyperlink to view either the Requests List, or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
4.
In the Occurrences column, click the number.
The Requests List popup screen opens, and displays all of the requests that triggered the learning suggestion.
On the View Full Request Information or View Request Information screens, you can view many details about the request such as:
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review the learning suggestions.
3.
In the Traffic Learning section, click a violation hyperlink to view either the request or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
The system displays the request or request elements that caused the learning suggestions for the selected violation.
4.
In the Occurrences column, if available, click the number.
The Requests List popup screen opens, and displays all of the requests that contained an item that triggered the learning suggestion.
Note: Some violations have no Occurrences number.
5.
In the Recent Incidents column (if attack signatures were detected), click the number.
The Requests List popup screen opens, and displays all of the requests that contained an item that triggered the learning suggestion.
6.
In the Requests List area of the popup screen, in the URL column, click a URL link.
The View Full Request Information screen or View Request Information opens in the popup screen, where you can review the request that triggered the learning suggestion.
7.
For each violation with a Learn button, click Learn to go back to the violation learning screen where you can accept or clear the learning suggestions for the security policy one value at a time.
8.
To view the actual contents of the request, click Full Request (on the View Request Information screen) or HTTP Request (on the View Full Request Information screen). and when you are done looking at the request details, click Close.
9.
On the screen showing learning suggestions for the violation, to accept the suggestion and change the security policy, click Accept.
10.
To remove learning suggestions without changing the security policy, select the ones to remove, and then click the Clear button.
11.
On the Manual Traffic Learning screen, continue to review the violations and associated learning suggestions.
If you want to review requests for a security policy that triggers learning suggestions, you can do so on the Requests screen.
1.
On the Main tab, expand Security, and click Event Logs.
The Event Logs: Application: Requests screen opens.
2.
Click Show Filter Details.
3.
For the Security Policy setting, select the name of the security policy for which you want to see requests.
4.
From the Request Type list, select All.
5.
Click the Go button.
The screen refreshes, and in the Requests List area, you see the requests for the selected security policy. Note that you only see staging suggestions if the logging profile for the security policy is set to log all requests.
Application Security Manager generates learning suggestions throughout the life of the security policy. When the system detects violations of a security policy, the violations may be related to a real attack, and may therefore warrant more careful inspection before being accepted into the security policy.
You can review learning suggestions (violations) on the Manual Traffic Learning screen, and accept or clear each suggestion, as described following. You can also view learning suggestions from the Enforcement Readiness Summary screen, as described in Using the Enforcement Readiness summary.
Note: When using automatic policy building to build a security policy, Policy Builder handles most learning suggestions by adjusting the policy. It is possible to see suggestions on the Traffic Learning screen even after the security policy is stable. You can review the suggestions and accept any that are caused by false positives.
The system provides learning suggestions for many of the violations. By default, learning suggestions are presented for the active policy. When you accept a learning suggestion, the system updates the current edited security policy to accept the request entity that triggered the violation.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
3.
Click a violation hyperlink.
The learning suggestions properties screen opens. Note that the screens vary for different violations.
4.
Select one or more learning suggestions, and then click the Accept, Apply, or Allow button, depending on the violation.
The system updates the security policy with the element in the request that caused the learning suggestion.
When you clear a learning suggestion, the system deletes the learning suggestion, and does not update the security policy. The system continues to generate learning suggestions for future instances of the violation.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
a)
Select one or more violations, and then click Clear.
A confirmation popup appears.
b)
Click OK.
The system deletes all of the learning suggestions and removes the violation from the list without changing the security policy.
a)
Click a violation hyperlink.
The violation properties screen opens.
b)
Select one or more learning suggestions, and then click Clear.
A confirm delete popup screen opens.
c)
Click OK.
The system deletes the learning suggestion without changing the security policy.
Note: For a description of the violation types, go to the Application Security: Blocking: Settings screen and click the next to the violation name. You can also refer to Appendix A, Security Policy Violations.
You use the Enforcement Readiness summary to review file types, URLs, parameters, cookies, and signatures that are in staging, and you can delve into the details to see if you want to add or update these entities in the security policy. You can add selected entities to the security policy, or you can enforce all of the entities that are ready to be enforced.
When you review the learning suggestions, you can clear them or go back to the enforcement readiness summary and enforce the entities. You can also click a learning suggestion in the list to have the security policy learn it, as described in Accepting a learning suggestion.
You can perform staging on file types, URLs, parameters, enforced cookies, and signatures to learn properties of entities, such as:
For URLs, learn meta characters (wildcard URLs only) and illegal content type violations including those associated with XML and JSON payloads
When an entity is in staging, the system does not block any requests for this entity. Instead, it posts learning suggestions for staged entities in the Violations Found for Staged Entities table in the request details.
Tip: Use staging on wildcard entities to build the security policy without specifying explicit entities of this type.
Staging is also useful when a site update occurs for a web application. Without staging, you might have to change the blocking policy enforcement mode to transparent for the entire web site to discover any new URLs or parameters in the updated web application. With staging, you can add any new URLs or parameters to the security policy, and place only the new entities in staging allowing the system to generate learning alerts.
If a file type, URL, parameter, or cookie is in staging or has learn explicit entities enabled, the system displays a status icon in the Staging or Learn Explicit Entities column of the file types, URLs, parameters, or cookies.
The icons in the Staging and Learn Explicit Entities columns provide details about the status of the file type, URL, or parameter. Move the cursor over the icon to see when the entity was placed in staging and the last time the properties of this entity were changed (the Last staging event time date and time).
On the Attack Signatures List screen, you can view the status of attack signatures that are in staging, as shown in Figure 12.2.
If the signature is in staging, the Learn column displays whether the signatures is in staging and for how long. For more information about attack signature staging, refer to Understanding attack signature staging.
After you create a security policy and traffic is sent to the web application, new entities are added by means of learn explicit entities, and existing entities are modified through staging. You can review the entities that are in staging and add the entities to the security policy. When the staging period is over and no learning suggestions are added for the staging period duration (the default is 7 days), the file type, URL, parameter, cookie, or signature is considered ready to be enforced. You can enforce the entities one at a time.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, and click Enforcement Readiness.
The Enforcement Readiness summary screen opens.
3.
In the Enforcement Readiness Summary, check to see if a number appears in the Not Enforced column.
A number greater than zero indicates that entities of that type are in staging or with learn explicit entities enabled.
4.
Click the number in the Not Enforced column.
The allowed file types, URLs, parameters, cookies, or signatures list opens showing the entities that you can enforce.
6.
Click Enforce.
The system takes the following actions:
To enforce all entities that are ready to be enforced
1.
On the Main tab, expand Security, point to Application Security, Policy Building, and click Enforcement Readiness.
The Enforcement Readiness summary screen opens.
4.
Click the Enforce Ready button.
The system takes the following actions:
Some of the violations are learnable meaning that the system can make learning suggestions about how to adjust the security policy when they occur. Other violations are unlearnable meaning that the system does not make learning suggestions for them because those violations do not concern issues that you want to change the policy to correct.
Note: Application Security Manager does not generate learning suggestions for requests that result in the web server returning HTTP responses with 400 or 404 status codes.
The following violations are considered learnable. The system suggests changes to the security policy when these violations occur.
F5 Networks recommends that you review the violations that occur, and consider whether they represent legitimate violations or false-positives. You can disable all violations if they are not applicable to your web application. However, F5 suggests disabling only unlearnable violations.
Disabling a violation turns off the blocking policy so that you are no longer notified of requests that trigger the violation. Alternately, you can clear the learning suggestions, and Application Security Manager continues to issue learning suggestions for the requests.
The Disable Violation button disables all flags on the selected violation. The system then ignores future instances of the violation, and passes the requests on to the web application resources. Be sure that you understand the ramifications of disabling a violation before doing it.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
4.
Click the Disable Violation button.
A confirmation popup screen opens.
5.
Click OK.
The screen refreshes, and you no longer see the violation in the Traffic Learning area.
Tip: You can navigate to the Application Security > Blocking Settings screen to see that all flags on the selected violation are unchecked.
6.
To put the security policy changes into effect immediately, click the Apply Policy button in the editing context area.
A confirmation popup screen opens.
7.
Click OK.
The system applies the updated security policy.
When you clear a violation, the system deletes the violation, but does not update the security policy. The system continues to generate alarms for future instances of the violation, and Application Security Manager continues to generate learning suggestions relative to the violation.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
3.
In the violations list, select the box next to a violation, and then click Clear.
A Confirm Delete popup screen opens.
4.
Click OK.
The system deletes the learning suggestion.
When you reject a learning suggestion for a URL, a file type, or a flow, the Application Security Manager adds the item to the ignored entities list. When the system receives subsequent requests for those items, the system no longer generates learning suggestions for them. The system does, however, continue to log the requests.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Manual Traffic Learning.
The Manual Traffic Learning screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review ignored entities.
3.
4.
5.
Select the Move to ignored entities check box and click OK.
The system adds the items you cleared to the ignored entities list.
For example, the following figure shows how when clearing an illegal file type, you have the choice to move the item to the ignored entities list.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Ignored Entities.
The Ignored Entities screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review ignored entities.
3.
On the Ignored Entities screen, if ignored entities exist for an entity type, that type becomes a link that you can click to view a list of all entities logged within that category. Click one of the links.
The Ignored File Types screen, Ignored URLs screen, or Ignored Flows screen opens.
4.
Select one or more entities, and then click Delete.
A Confirm Delete popup screen opens.
5.
Click OK.
The system removes the selected ignored entities from the ignored item status.
If you want the system to start generating learning suggestions for items that were previously added to the ignored entities list, you can remove those items from the list.
1.
On the Main tab, expand Security, point to Application Security, Policy Building, then click Ignored Entities.
The Ignored Entities screen opens.
2.
In the editing context area, ensure that the current edited security policy is the one for which you want to review ignored entities.
3.
Select the entity type whose ignored entities you want to remove, and click the Delete button.
The system removes all ignored items of the selected entity type from the ignored item status and resumes generating learning suggestions for this entity type.
For each security policy, you can create a centralized list of IP address exceptions, or IP addresses that the system should treat differently. You can specify that the system trusts certain IP addresses, never blocks or never logs traffic sent from these IP addresses, and ignores certain IP addresses in anomaly detection. You can also instruct the system not to generate learning suggestions for traffic sent from these IP addresses.
Creating a list of IP address exception is useful, for example, if your company performs penetration testing using manual or automatic scanners. When you add the IP address of the scanner, you can prevent the system from generating learning suggestions for traffic from the scanner, but still have the system make learning suggestions for other legitimate production traffic.
1.
On the Main tab, expand Security, point to Application Security, IP Addresses, then click IP Address Exceptions.
The IP Address Exceptions screen opens.
2.
In the editing context area, ensure that the current edited web application is the one for which you want to add IP address exceptions.
3.
Click the Create button.
The New IP Address Exception screen opens.
5.
To instruct the system to always trust this IP address, for the Policy Builder trusted IP setting, select the Enabled check box.
If you enable this setting, the Policy Builder automatically adds the traffic data from this IP address to the security policy. The system adds this IP address to the Trusted IP Addresses list on the Application Security: Policy Building: Settings screen.
6.
To have the system ignore this IP address when performing brute force prevention and web scraping detection, for the Ignore in Anomaly Detection setting, select the Enabled check box.
If you enable this setting, the system automatically adds this IP address to the IP Address Whitelists on the brute force and web scraping screens.
7.
If you do not want the system to generate security policy suggestions for traffic from this IP address, for the Ignore in Learning Suggestions setting, select the Enabled check box.
8.
To prevent the system from blocking traffic from this IP address, for the Never Block this IP Address setting, select the Enabled check box.
9.
To instruct the system not to log requests from this IP address, for the Never log requests from this IP Address setting, select the Enabled check box.
If you enable this setting, the system does not log requests sent from this IP address, even if the traffic is illegal, and even if your security policy is configured to log all traffic.
10.
If you want the system to consider this IP address legitimate even if it is in the IP address intelligence database, for the Ignore IP Address Intelligence setting, select the Enabled check box.
11.
In the Description field, type a note about why this IP address is an exception.
12.
Click Create.
The system adds the IP address to the list of IP address exceptions.
1.
On the Main tab, expand Security, point to Application Security, IP Addresses, then click IP Address Exceptions.
The IP Address Exceptions screen opens.
2.
In the editing context area, ensure that the current edited web application is the one with IP address exceptions you want to change.
3.
Select the IP address exception that you want to remove, and click the Delete button.
After you confirm, the IP address is removed from the IP address exceptions list (and also from the any other lists it was on such as the trusted IP address list and the anomaly detection whitelists.
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?




NOTE: Please do not provide personal information.



Incorrect answer. Please try again: Please enter the words to the right: Please enter the numbers you hear:

Additional Comments (optional)