Manual Chapter : Adding URLs to a Security Policy

Applies To:

Show Versions Show Versions

BIG-IP ASM

  • 11.5.10, 11.5.9, 11.5.8, 11.5.7, 11.5.6, 11.5.5, 11.5.4, 11.5.3, 11.5.2, 11.5.1
Manual Chapter

About adding URLs

In a security policy, you can manually specify the file types that are allowed (or disallowed) in traffic to the web application being protected. This is only if you are not using automatic policy building which F5 recommends doing. When using automatic policy building, Application Security Manager determines which file types to add, based on legitimate traffic.

When you create a security policy, a wildcard file type of *, representing all file types, is added to the file type list. During the enforcement readiness period, the system examines the file types in the traffic and makes learning suggestions that you can review and add the file types to the policy as needed. This way, the security policy includes the file types that are typically used. When you think all the file types are included in the security policy, you can remove the * wildcard from the allowed file types list.

About referrer URLs

Referrer URLs are web pages that request other URLs within a web application. For example, an HTML page can request a GIF, JPG, or PNG image file. The HTML page is the referrer, and the GIF, JPG, and PNG files are non-referrers. In lists of URLs, non-referrer URLs appear in blue and referrer URLs appear in gold.

A referrer in Application Security Manager is similar to the HTTP Referer header. Use referrers for complex objects, such as HTML pages, but not for embedded objects, such as GIF files.

Adding allowed URLs

You can manually add allowed URLs, which are URLs that the security policy accepts in traffic to the web application being protected.
  1. On the Main tab, click Security > Application Security > URLs. The Allowed URLs screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. Click Create. The New Allowed URL screen opens.
  4. For URL, choose a type and protocol, and then type the URL name or wildcard.
    Option Description
    Explicit Specifies a unique URL, such as /index.html. Choose HTTP or HTTPS, and type the URL in the adjacent field.
    Wildcard Specifies that the URL is a wildcard expression. Any URL that matches the wildcard expression is considered legal. The pure wildcard (*) is automatically added to the security policy so you do not need to add it. But you can add other wildcards such as /main/*. Select HTTP or HTTPS, and type a wildcard expression in the adjacent field.
  5. By default, the Perform Staging check box is selected. F5 recommends that you keep it selected unless you are creating a wildcard URL for which you plan to Add All Entities in the Learn Explicit Entities setting. In that case, clear it.
  6. If you are creating a wildcard URL, using the Learn Explicit Entities list, specify whether the system adds explicit URLs that match a wildcard to the security policy.
    Option Description
    Never (wildcard only) The system does not add or suggest that you add entities that match the wildcard to the policy. When false positives occur, the system suggests relaxing the settings of the wildcard entity. This option results in a security policy that is easy to manage but may not be as strict.
    Add All Entities The system creates a comprehensive whitelist policy that includes all of the website entities. This option will form a large set of security policy entities, which produce a granular object-level configuration and high security level; it may take more time to maintain such a policy.
    Note: Do not enable both staging and Add All Entities on the same wildcard entity.
  7. If you want to view more options, next to Create New Allowed URL, select Advanced.
  8. To process requests for this URL according to the header content, create header-based content profiles. Another task provides details on how to do this.
  9. To protect the application from being able to harbor illegitimate frames and iframes with malicious code in the application, set up protection from clickjacking:
    1. For Clickjacking Protection, select the Enabled check box.
    2. From the Allow Rendering in Frames list, select an option to determine whether to allow this URL to be rendered in a frame or iframe.
  10. For wildcard URLs, leave Wildcard Match Includes Slashes selected. When this option is selected, an asterisk in a wildcard matches any number of path segments (separated by slashes); when cleared, an asterisk matches at most one segment.
  11. For wildcard URLs, in the Meta Characters tab, you can specify whether to check for meta characters in the URL, and which ones to allow or disallow.
    1. The Check characters on this URL setting is enabled by default so that the system verifies meta characters in the URL. (If you do not want to check for meta characters, clear the check box and skip to the next step.)
    2. To specify which meta characters to allow or disallow, from the Global Security Policy Settings list, select any meta characters that you want to specifically allow or disallow, and move them to the Overridden Security Policy Settings list.
    3. For each meta character that you moved, set the state to Allow or Disallow.
    Note: The Overridden Security Policy Settings take precedence over the global settings for the web application character set.
  12. Click Create. The Allowed URLs screen opens and lists the new URL.
  13. To put the security policy changes into effect immediately, click Apply Policy.
The security policy allows requests for the URL or URLs matching the wildcard that you added. If the URL is in staging, the system informs you when learning suggestions are available or when it is ready to be enforced.

Wildcard syntax

The syntax for wildcard entities is based on shell-style wildcard characters. This table lists the wildcard characters that you can use in the names of file types, URLs, parameters, or cookies so that the entity name can match multiple objects.

Wildcard Character Matches
* All characters
? Any single character
[abcde] Exactly one of the characters listed
[!abcde] Any character not listed
[a-e] Exactly one character in the range
[!a-e] Any character not in the range

Allowed URL properties

These tables describe the allowed URL properties (both Basic and Advanced settings) that appear on different parts of the screen.

Create New Allowed URL properties

Property Description
URL Specifies a URL that the security policy allows. The available types are:
  • Explicit: Specifies that the URL is a unique URL. Type the URL in the adjacent field using the format /index.html.
  • Wildcard: Specifies a wildcard expression. Any URL that matches is considered legal. For example, typing * specifies that any URL is allowed by the security policy. Type a wildcard expression in the adjacent field.
Protocol Specifies whether the protocol for the URL is HTTP or HTTPS.
Perform Staging Specifies that the system places this URL in staging. Learning suggestions produced by requesting staged URLs are logged in the Learning screens. Review staging status on the URL List screen. If a URL is in staging, point to the icon to display staging information.When you are no longer getting learning suggestions, you can disable this setting. If you enforce a URL, this setting is cleared.
Learn Explicit Entities Specifies how to add or suggests you add URLs to the security policy if you are creating a wildcard URL.
  • Add All Entities: The system suggests you add explicit URLs that match the wildcard to the security policy creating a comprehensive whitelist of all the URLs on the web site. You can review suggestions on the Traffic Learning screen.
  • Never (wildcard only): The system does not add URLs that match the wildcard to the security policy, and suggests changing the attributes of matched wildcard entities.
Check Flows to this URL Specifies that the security policy validates flows to the URL (if configured). If this setting is disabled, the system ignores the flows to the URL. When you select this check box, additional settings appear.
URL is Entry Point (Visible when Check Flows to this URL is selected.) Specifies that this URL is a page through which a visitor can enter the web application.
URL is Referrer (Visible when Check Flows to this URL is selected.) Specifies that the URL is a URL from which a user can access other URLs in the web application.
URL can change Domain Cookie Specifies that the security policy does not block an HTTP request where the domain cookie was modified on the client side. Note that this setting is applicable only if the URL is a referrer.
URL with Navigation Parameter Specifies that you want to associate a navigation parameter with this URL. You must have a navigation parameter defined in the security policy to view this option.
Select Navigation Parameter Specifies a list of navigation parameters that you can associate with this URL.
Navigation Parameter Value Indicates the value of the navigation parameter.
Clickjacking Protection Specifies that the system adds the X-Frame-Options header to the domain cookie’s response header. This is done to protect the web application against clickjacking. Clickjacking occurs when attacker lures a user to click illegitimate frames and iframes because the attacker hid them on legitimate visible website buttons. Therefore, enabling this option protects the web application from other web sites hiding malicious code behind them. The default is disabled. After you enable this option, you can select whether, and under what conditions, the browser should allow this URL to be rendered in a frame or iframe.
Allow Rendering in Frames Specifies the conditions for when the browser should allow this URL to be rendered in a frame or iframe.
  • Never: Specifies that this URL must never be rendered in a frame or iframe. The web application instructs browsers to hide, or disable, frame and iframe parts of this URL.
  • Same Origin Only: Specifies that the browser may load the frame or iframe if the referring page is from the same protocol, port, and domain as this URL. This limits the user to navigate only within the same web application.
  • Only From URL: Specifies that the browser may load the frame or iframe from a specified domain. Type the protocol and domain in URL format - for example, http://www.mywebsite.com. Do not enter a sub-URL, such as http://www.mywebsite.com/index.
Wildcard Match Includes Slashes Specifies that an asterisk in a wildcard URL matches any number of path segments (separated by slashes); when cleared, specifies that an asterisk matches at most one segment. For example: the wildcard /art/* matches /art/abc/index.html if the wildcard match includes slashes (default value), but does not match it if the check box is cleared. In that case, it matches /art/go.html (only one segment below /art).
HTML5 Cross-Domain Request Enforcement CORS (Cross-Origin Resource Sharing) lets one website access the resources of another website using JavaScript (within the browser). Web applications may share resources with other websites hosted on a different domain. When the option is selected, the system protects a specific URL in your web application from cross-origin resource sharing. You can configure which domains can access the response generated by requesting this URL (the resource), and how to overwrite CORS response headers returned by the web server.
URL Description Describes the URL (optional).

Header-Based Content Profiles

Property Description
Request Header Name Specifies an explicit header name that must appear in requests for this URL. This field is not case-sensitive.
Request Header Value Specifies a simple pattern string (glob pattern matching) for the header value that must appear in legal requests for this URL; for example, *json*, xml_method?, or method[0-9]. If the header includes this pattern, the system assumes the request contains the type of data you select in the Request Body Handling setting. This field is case-sensitive.
Request Body Handling Indicates how the system parses the content of requests for the allowed URL:
  • Apply Content Signatures: Do not parse the content; scan the entire payload with full-content attack signatures.
  • Apply Value and Content Signatures: Do not parse the content or extract parameters; process the entire payload with value and full-content attack signatures.
  • Disallow: Block requests for an URL containing this header content. Log the Illegal Request Content Type violation.
  • Do Nothing: Do not inspect or parse the content. Handle the header of the request as specified by the security policy.
  • Form Data: Parse content as posted form data in either URL-encoded or multi-part formats. Enforce the form parameters according to the policy.
  • GWT: Perform checks for data in requests, based on the configuration of the GWT (Google Web Toolkit) profile associated with this URL.
  • JSON: Review JSON data using an associated JSON profile, and use value attack signatures to scan the element values.
  • XML: Review XML data using an associated XML profile.
Profile Name Specifies the XML, JSON, or GWT profile the security policy uses when examining requests for this URL if the header content is parsed as XML, JSON, or GWT. You can also create or view the XML, JSON, or GWT profile from this option.

HTML5 Cross-Domain Request Enforcement

Property Description
Allow HTML5 Cross-Origin Requests Allows all CORS requests to this URL, and displays additional settings.
Allowed Origins Allows you to specify a list of origins allowed to share data returned by this URL.
Allowed Methods Allows you to specify a list of methods that other web applications hosted in different domains can use when requesting this URL.
Allowed Headers Allows you to specify a list of request headers that other web applications hosted in different domains can use when requesting this URL. Or you can delete non-simple headers returned in response to requests.
Exposed Headers Allows you to specify a list of response headers that are safe to expose to JavaScript, and can be shared with web applications hosted in different domains.Or you can allow only simple headers to be exposed.
Allow Credentials Specifies whether requests from other web applications hosted in different domains may include user credentials.
Maximum Age Specifies how long (in seconds) to cache in the browser the results of a preflight request (a special request that the browser sends to your web application to determine if JavaScript from another domain may access your resource).

Meta Characters

Property Description
Check characters on this URL Specifies that the system verifies meta characters on this URL. You can change which meta characters are allowed or disallowed.

Adding disallowed URLs

For some web applications, you may want to deny requests for certain URLs. In this case, you can create a set of disallowed URLs. Adding disallowed URLs is useful, for example, to prevent access to an administrative interface to the web application such as /admin/config.php.
  1. On the Main tab, click Security > Application Security > URLs > Disallowed URLs. The Disallowed URLs screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. Click Create. The New Disallowed URL screen opens.
  4. For the URL (Explicit only) setting, select HTTP or HTTPS as the protocol for the URL, and type the URL that the security policy considers illegal; for example, /index.html.
    Note: URLs are case-sensitive unless you cleared the Security Policy is case sensitive option when you created the policy.
  5. Click Create. The Disallowed URLs screen opens and lists the URL.
  6. To put the security policy changes into effect immediately, click Apply Policy.
If a requested URL is on the disallowed URLs list, the system ignores, learns, logs, or blocks the request depending on the settings you configure for the Illegal URL violation on the Application Security: Blocking: Settings screen. You can view learning suggestions for disallowed URLs on the Illegal URL learning screen.

Enforcing requests for URLs based on header content

Before you can enforce requests for URLs using header content, you need to have added an allowed URL.
When you manually create a new allowed URL, the system reviews requests for the URL using HTTP parsing. The system automatically creates a default header-based content profile for HTTP, and you cannot delete it. However, requests for an URL may contain other types of content, such as JSON, XML, or other proprietary formats.

You can use header-based content profiles to configure how the system recognizes and enforces requests for this URL according to the header content in the request. You can also use header-based content profiles to block traffic based on the type of header and header value in requests for a URL.

  1. On the Main tab, click Security > Application Security > URLs. The Allowed URLs screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Allowed URLs List, click the name of the URL for which you want to specify legal header content. The Allowed URL Properties screen opens where you can modify the properties of the URL.
  4. From the Allowed URL Properties list, select Advanced.
  5. In the Header-Based Content Profiles area, specify the header and value as follows:
    1. In the Request Header Name field, type the explicit header name that must appear in requests for this URL. This field is not case-sensitive.
    2. In the Request Header Value field, type the pattern string for the header value to find in legal requests for this URL, for example, *json*, xml_method?, or method[0-9]. This field is case-sensitive. If a header value includes this pattern, the system assumes that the request contains the type of data you select in the Request Body Handling setting.
    3. From the Request Body Handling list, specify how the system recognizes and enforces requests for this URL according to the requests’ header content:
      Option Result
      Apply Content Signatures Do not parse the content; scan the entire payload with full-content attack signatures.
      Apply Value and Content Signatures Do not parse the content or extract parameters; process the entire payload with value and full-content attack signatures. This option provides basic security for protocols other than HTTP, XML, JSON, and GWT; for example, use *amf* as the header value for a content-type of Action Message Format.
      Disallow Block requests for an URL containing this header content. The system logs the Illegal Request Content Type violation.
      Do Nothing Do not inspect or parse the content. Handle the header of the request as specified by the security policy.
      Form Data Parse content as posted form data in either URL-encoded or multi-part formats. Enforce the form parameters according to the policy.
      GWT Examine data in requests, based on the configuration of a GWT (Google Web Toolkit) profile associated with this URL.
      JSON Examine JSON data using an associated JSON profile, and use value attack signatures to scan the element values.
      XML Examine XML data using an associated XML profile.
    4. If the content is GWT, JSON, or XML, select an existing profile or click the create (+) button to create one. (The other options do not require special profiles.)
    5. Click Add.
  6. Click Update.
  7. To put the security policy changes into effect immediately, click Apply Policy.
If the system detects a request for a URL, which contains header content that is disallowed in the URL's Header-Based Content Profile, the Illegal request content type violation occurs.

Specifying characters legal in URLs

When you create a security policy, you select a language encoding (or let the system determine it automatically) that determines the characters that can be used in URLs. You can view or modify which characters the security policy allows or disallows in URLs.
  1. On the Main tab, click Security > Application Security > URLs > Character Set. The URLs Character Set screen opens, where you can view the character set, and state of each character.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. Use the View option to display the characters that you want to see.
    Tip: To restore the default character set definitions, you can click the Restore Defaults button at any time.
  4. To modify which characters the system should permit or prohibit in the name of a wildcard URL, click Allow or Disallow next to the character.
  5. Click Save to save the changes.
  6. To put the security policy changes into effect immediately, click Apply Policy.
The security policy checks that URLs in requests do not include any disallowed characters. For example, by disallowing the characters <, >, ', and |, a security policy protects against many cross-site scripting attacks and injection attacks.

Configuring flows to URLs

Before you can configure a flow, should have created the explicit URL for which you want to add the flow.
A flow defines the access path leading from one explicit URL to another, between a referrer URL and a target URL in a web application. For example, a basic web page may include a graphic and hyperlinks to other pages in the application. The calls from the basic page to the other pages make up the flow. You can configure flows to a URL.
Note: Configuring flows is an optional task. Unless you need the enhanced security of configured flows, F5 Networks recommends that you do not configure flow-based security policies due to their complexity.
  1. On the Main tab, click Security > Application Security > URLs. The Allowed URLs screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Allowed URLs List, click the name of the URL you want to modify. The Allowed URL Properties screen opens.
  4. On the menu bar, click Flows to URL. The Flows to URL screen opens and shows the flows to that specific URL.
  5. Above the Flows to URL area, click Create. The Create a New Flow popup screen opens.
  6. For the Referrer URL setting, specify how the client enters the application.
    • If you want the client to enter the application from this URL, select Entry Point.
    • To specify the path of a referrer URL that refers to other URLs in the application, select URL Path; for example, type /index.html.
  7. From the Protocol list, select the appropriate protocol, HTTP or HTTPS.
  8. From the Method list, select the HTTP method that the URL expects a visitor to use to access the authenticated URL, for example, GET or POST.
  9. In the Frame Target field, type the index (0-29, or 99) of the HTML frame in which the URL belongs, if the web application uses frames.
    Tip: If your web application does not use frames, type the value 1.
  10. If this flow can contain a query strings or POST data, select the Allow QS/PD check box.
  11. If you want the system to verify query strings or POST data for this flow, select the Check QS/PD check box.
  12. Click OK. The popup screen closes, and on the Flows to URL screen, you see the URLs from which the URL can be accessed.
  13. To view the flow or modify the flow properties, click the URL in the Flows list.
  14. To view the entire application flow, click Security > Application Security > URLs > Flows List
  15. To put the security policy changes into effect immediately, click Apply Policy.
You now have the option of creating parameters that are associated with the flow.

Creating flow parameters

Before you can create a flow parameter, you need to first have created the flow to which the parameter applies. If the source URL is a referrer URL, that URL must already be defined in the security policy as well.
You define parameters in the context of a flow when it is important to enforce that the target URL receives a parameter only from a specific referrer URL. Flow parameters provide very tight, flow-specific security for web applications. With this increased protection comes an increase in maintenance and configuration time. Note that if your application uses dynamic parameters, you need to manually add those to the security policy.
  1. On the Main tab, click Security > Application Security > Parameters.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. Click Create. The Add Parameter screen opens.
  4. In the Create New Parameter area, for the Parameter Name setting, specify the type of parameter you want to create.
    • To create a named parameter, select Explicit, then type the name.
    • To use pattern matching, select Wildcard, then type a wildcard expression. Any parameter name that matches the wildcard expression is permitted by the security policy.
    • To create an unnamed parameter, select No Name. The system creates a parameter with the label, UNNAMED.
  5. In the Parameter Level setting, select Flow, and then for From URL define where the flow must come from:
    • If the source URL is an entry point, click Entry Point.
    • If the source URL is a referrer URL (already defined in the policy), click URL Path, select the protocol used for the URL, then type the referrer URL associated with the flow.
    When you begin to type the URL, the system lists all referrer URLs that include the character you typed, and you can select the URL from the list.
  6. In the Parameter Level setting, for Method, select the HTTP method (GET or POST) that applies to the target referrer URL (already defined in the policy).
  7. In the Parameter Level setting, for To URL, select the protocol used for the URL, then type the target URL.
  8. Leave the Perform Staging check box selected if you want the system to evaluate traffic before enforcing this parameter. Staging helps reduce the occurrence of false positives.
  9. If you are creating a wildcard parameter and you want the system to display explicit parameters that match the wildcard entity pattern that you specify, for the Learn Explicit Entities setting, select Add All Entities.
  10. If the parameter is required in the context of the flow, select the Is Mandatory Parameter check box. Note that only flows can have mandatory parameters.
  11. Specify whether the parameter requires a value:
    • If the parameter is acceptable without a value, leave the Allow Empty Value setting enabled.
    • If the parameter must always include a value, clear the Allow Empty Value check box.
  12. To allow users to send a request that contains multiple parameters with the same name, select the Allow Repeated Occurrences check box.
    Important: Before enabling this check box, consider that requests containing multiple parameters of the same name could indicate an attack on the web application (HTTP Parameter Pollution).
  13. If you want to treat the parameter you are creating as a sensitive parameter (data not visible in logs or the user interface), enable the Sensitive Parameter setting.
  14. For the Parameter Value Type setting, select the format of the parameter value. Depending on the value type you select, the screen refreshes to display additional configuration options.
  15. Click Create to add the new parameter to the security policy.
  16. To put the security policy changes into effect immediately, click Apply Policy.
When you create a parameter that is associated with a flow, the system verifies the parameter in the context of the flow. For example, if you define a parameter in the context of a GET request, and a client sends a POST request that contains the parameter, the system generates an Illegal Parameter violation.

Configuring dynamic flows to URLs

Before you can configure a dynamic flow, you must have created the explicit URL for which you want to add the dynamic flow.
Some web applications contain URLs with dynamic names, for example, the links to a server location for file downloads, where the file name may be unique to each user. You can configure the system to detect these URLs by creating a dynamic flow. For the dynamic flow, you specify a regular expression that describes the dynamic name, and associate the flow with the URL.
  1. On the Main tab, click Security > Application Security > URLs. The Allowed URLs screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Allowed URLs List, click the name of the URL you want to modify. The Allowed URL Properties screen opens.
  4. On the menu bar, click Dynamic Flows from URL. The Flows to URL screen opens and shows the flows to that specific URL.
  5. Above the Dynamic Flows to URL area, click Create. The Create a New Dynamic Flow popup screen opens.
  6. In the Prefix field, type a fixed substring that appears near the top of the HTML source page before the dynamic URL. The prefix may be the name of a section in combination with HTML tags, for example, <title>Online Banking</title>.
  7. For the RegExpValue setting, type a regular expression that specifies the set of URLs that make up the dynamic flow, for example, a set of archive files.
  8. For the Suffix setting, type a fixed string that occurs in the referring URL’s source code, and is physically located after the reference to the dynamic name URL.
  9. Click OK. The popup screen closes, and on the Dynamic Flows from URL screen, you see the dynamic flow extraction properties.
  10. To put the security policy changes into effect immediately, click Apply Policy.
The regular expression describes the dynamic URL name. The Application Security Manager extracts dynamic URL names from the URL responses associated with the dynamic flow.

Configuring dynamic session IDs in URLs

If an application uses dynamic information in URLs (for example, user names), the Application Security Manager cannot use its normal functions to extract and enforce URLs or flows because the URI contains a dynamic element. If the web application that you are securing could contain dynamic information in a URL, you can allow dynamic session IDs in URLs. (You only need to configure this setting if you know that your application works this way.)
  1. On the Main tab, click Security > Application Security > Security Policies. The Active Policies screen opens.
  2. Click the name of the security policy you want to work on. The Properties screen opens.
  3. From the Configuration list, select Advanced.
  4. For the Dynamic Session ID in URL setting, specify how the security policy should process URLs that use dynamic sessions.
    Use this option When
    Custom pattern The security policy uses a user-defined regular expression to recognize a dynamic session ID in URLs. Type a regular expression in the Value field, and a description in the Description field.
    Default pattern The security policy uses the default regular expression (\/sap\([^)]+\)) for recognizing a dynamic session ID in URL.
    Disabled The security policy does not enforce dynamic session IDs in URLs. This is the default value.
  5. Click Save to save your settings.
  6. To put the security policy changes into effect immediately, click Apply Policy.

Normally, if the system receives a request in which the dynamic session information does not match the settings in the security policy, the system issues the Illegal session ID in URL violation. When you allow dynamic session IDs in URLs, ASM extracts the dynamic session information from requests or responses, based on the pattern that you configure. For requests, the system applies the pattern to the URI up to, but not including, the question mark (?) character in a query string.

Note: The system can extract dynamic information only from illegal URLs.