You can use learning resources to help build a security policy, particularly if you are building a security policy manually. When building a security policy manually, the learning mode is set to Manual, and when building a policy automatically, the learning mode is Automatic.
When you send client traffic through the Application Security Manager™ (ASM), the learning data provides information on requests or responses that do not comply with the current security policy and have triggered a violation. The reason for triggering a violation can be either an actual attack on the site, or a false positive (typically seen during the process of building a policy).
ASM™ generates learning suggestions for requests that cause violations and do not pass the security policy checks. The system also suggests adding legitimate entities such as URLs, file types, or parameters that often appear in requests. You can examine the requests that cause learning suggestions, and then use the suggestions to refine the security policy. In some cases, learning suggestions may contain recommendations to relax the security policy. When dealing with learning suggestions, make sure to relax the policy only where false positives occurred, and not in cases where a real attack caused a violation. You can use the violation ratings to help determine how likely a request was caused by an attack.
If you are generating a security policy automatically, ASM handles much of the learning for you, adjusting the security policy based on traffic characteristics. In that case, the learning screens show only the elements that the security policy is in the process of learning, or those which require manual intervention to be resolved.
Application Security Manager™ (ASM) generates learning suggestions for violations if the Learn flag is enabled for the violations on the Learning and Blocking Settings screen. When the system receives a request that triggers a violation, the system updates the Traffic Learning screen with learning suggestions using information from the violating request. From this screen, you can review the learning suggestions to determine whether the request triggered a legitimate security policy violation, or if the violation represents a need to update the security policy.
The system can also generate suggestions based on legitimate activity, such as adding a valid URL or host name to the security policy.
Next to each suggestion, ASM assigns a learning score that measures the strength of the suggestion by showing a percentage that indicates how close the system is to recommending that you accept the suggestion. The learning score is also influenced by the violation rating: the lower the rating of the violations, the higher the score.
If the system is working in automatic learning mode, when the learning score reaches 100%, the system accepts and enforces most of the suggestions, or you can accept suggestions manually at any time. If you are using manual learning, when the learning score reaches 100% (or before that if you know the suggestions are valid), you need to accept the suggestions manually.
Making decisions about which learning suggestions to accept requires a general understanding of application security, and specific knowledge of the protected application (for example, recognizing valid traffic). For example, you should consider accepting a learning suggestion when you see that it is associated with many requests from many different source IP addresses. As long as they are valid, repeated requests may indicate legitimate traffic behavior that warrants relaxing the security policy.
You can also review the violation rating for requests by selecting the suggestion. Learning suggestions associated with requests having a low average violation rating are more likely to be false positives and can be accepted. But if a request has a high violation rating, the learning suggestion should not be accepted. Instead, it should be cleared because it is most likely indicative of an attack.
The Traffic Learning screen also displays violations for which the system does not generate learning suggestions. Typically, these violations are related to RFC compliance and system resources; the resolution for these violations may be to disable the violation rather than to change the configuration. The system displays these violations along with the learning suggestions to ease the security policy management tasks.
This figure shows the Traffic Learning screen with several suggestions on it. As an example, on the left, the suggestion to enforce a cookie is highlighted; the information on the right shows what caused the suggestion. The HTTP violations are listed, and one is selected showing details about the request. The cookie PHPAUCTION_SESSION matched the * wildcard in the Allowed Cookies list in several requests so the suggestion is to add and enforce the cookie. If you accept the suggestion, the cookie is added to the Enforced Cookies list.
Traffic Learning screen with suggestions
Some violations that occur indicate a real problem with a request that cannot be learned. These are called unlearnable violations. For example, requests for access from disallowed users, disallowed sessions, and disallowed IP addresses are unlearnable. In addition, the system considers requests that trigger the following HTTP protocol compliance violations to be unlearnable:
They are considered unlearnable because these violations indicate behavior that is never acceptable, so the security policy will never be changed to allow them. Consequently, the violating requests are not used for automatic or manual learning (even if they include additional violations that could be learned). No learning suggestions are created for requests containing these violations. Also, the violation rating for these transactions is always set to 5 (the highest severity).
You can adjust the learning settings for file types, URLs, parameters, cookies, and redirection domains. Learning settings specify when Real Traffic Policy Builder® adds, or suggests you add, explicit entities to the security policy.
|Never (wildcard only)||Specifies that when false positives occur, the system suggests relaxing the settings of the wildcard. This option results in a security policy that is easy to manage, but is not as strict. If it is running in automatic learning mode, the Policy Builder does not add explicit entities that match a wildcard to the security policy. The wildcard entity remains in the security policy. The Policy Builder changes the attributes of any matched wildcard. If it is running in manual learning mode, Policy Builder suggests changing the attributes of matched wildcard entities, but does not suggest that you add explicit entities that match the wildcard entity.|
|Selective||Applies only to * wildcard entity. When false positives occur, adds an explicit entity with relaxed settings. This option serves as a good balance between security, policy size, and ease of maintenance. If using automatic learning, the policy includes explicit entities that do not match the attributes of the * wildcard, and does not remove the * wildcard. If usingmanuial learning, the system suggests adding explicit entities that match the * wildcard. (This option is not available for redirection protection.)|
|Compact|| Applies only to the * wildcard. Specifies that the policy includes
the most commonly used entities (while enforcing all others types with a
wildcard rule), also provides a pre-populated list of known disallowed
file types, and includes top-level URLs such as /abc/*. This option
serves as a good balance between Selective and
Always making a smaller, more compact policy,
with fewer suggestions.
When using Automatic learning, the system adds explicit entities that do not exist in the policy but which match the attributes of the * wildcard. The Policy Builder does not remove the * wildcard file type from the security policy. For Manual learning, the system suggests adding explicit entities that match the * wildcard file type. (This option is not available for cookies or redirection protection.)
|Always||Creates a comprehensive whitelist policy that includes all web site entities. This option results in a large, more granular configuration with stricter security. If Policy Builder is running, it adds explicit entities that match a wildcard to the security policy. When the security policy is stable, the * wildcard is removed. If Policy Builder is not running, the system suggests adding explicit entities that match the wildcard. (This option is not available for cookies.)|
The security policy now learns new file types, parameters, URLs, cookies, and redirection domains according to the learning settings you specified.
If you are using automatic learning to build a policy, you can have the system examine responses as well as requests for entities to include in the security policy. This is called learning from responses, and the system does this by default. You may want to learn from responses because a response might include more information about the web application than is found in the request, or if you want to have the system learn login pages automatically.
You can disable this setting if your application does not need to examine responses for entities to add to the security policy, or if the application does not use dynamic parameters.
If you disabled the Learn from responses check box, the Policy Builder never adds to the security policy elements found in responses. If the check box is enabled, the Policy Builder adds elements found in valid responses to the security policy (meaning those that do not generate violations).
When using automatic or manual learning, the system learns from legitimate traffic including transactions that return response codes of 1xx, 2xx, and 3xx. These classes of codes are added by default to the policy building settings. You can change which response codes are listed, or add specific response codes, such as those used by the web application you are protecting.
|1xx||All informational responses (the request was received; continuing to process it). Included by default.|
|2xx||All successful responses (the request was received, understood, accepted, and processed successfully). Included by default.|
|3xx||All redirection (the client needs to take additional action on the request). Included by default.|
|4xx||Server failed to fulfill the response as a result of client syntax or input errors.|
|5xx||All server error responses (the server failed to fulfill a request).|
|Specific codes such as 100, 306, 400, or 404||Refer to your web application or the Hypertext Transfer Protocol -- HTTP/1.1 specification (RFC-2616).|
After you create a security policy and begin sending traffic to the application, the system provides learning suggestions concerning additions to the security policy based on the traffic it sees. For example, you can have users or testers browse the web application. By analyzing the traffic to and from the application, Application Security Manager™ generates learning suggestions or ways to fine-tune the security policy to better suit the traffic and secure the application.
|Accept Suggestion||The system modifies the policy by taking the suggested action, such as adding an entity that is legitimate. If the entity that triggered the suggestion can be placed in staging (file types, URLs, parameters, cookies, or redirection domains), clicking Accept Suggestion displays a second option, Accept suggestion and enable staging on Matched <<entity>>. Click this option to accept the suggestion and place the matched entity in staging.|
|Delete Suggestion||The system removes the learning suggestion, but the suggestion reoccurs if new requests cause it. The learning score of the suggestion starts over from zero in that case.|
|Ignore Suggestion||The system does not change the policy and stops showing this suggestion on the Traffic Learning screen now and in the future. You can view ignored suggestions by filtering by status ignored.|
If you know that a suggestion is valid, you can accept it at any time even before the learning score reaches 100%. The ones that reach 100% have met all the conditions so that they are probably legitimate entities.
When you are creating a security policy, you specify an enforcement readiness period that indicates a staging period for entities and attack signatures (typically 7 days). When entities or attack signatures are in staging, the system does not enforce them. Instead, the system posts learning suggestions for staged entities.
When the enforcement readiness period is over and no learning suggestions are added for the staging period duration (the default is 7 days), the file type, URL, parameter, cookie, signature, or redirection domain is considered ready to be enforced. Particularly if you are using manual learning, you can delve into the details to see if you want to enforce these entities in the security policy. From the Enforcement Readiness summary on the Traffic Learning screen, you can enforce selected entities to the security policy, or you can enforce all of the entities and signatures that are ready to be enforced. If you are using automatic learning, you can still enforce entities manually, but the Policy Builder enforces entities according to the learning and blocking settings. So you do not need to enforce entities in the security policy.
Even though you are done creating a security policy, Application Security Manager™ (ASM) might have additional action items it recommends for you to do based on your current system configuration and current security policies.