Manual Chapter : Customizing URL Categories and Filters for SWG

Applies To:

Show Versions Show Versions

BIG-IP APM

  • 13.1.5, 13.1.4, 13.1.3, 13.1.1, 13.1.0
Manual Chapter

Overview: Customizing URL categories and filters for SWG

On a BIG-IP® system with an SWG subscription, you can customize URL categories and URL filters any time after the initial download of the URL database has completed. Customizing URL categories and URL filters is completely optional.

With regularly scheduled downloads, URLs are added to the URL database on an ongoing basis. With predefined URL filters, if they completely serve your needs, you do not need to configure more.

About the Instant Messaging URL category

Note: A predefined Instant Message URL category is available only on a BIG-IP® system with an SWG subscription.

Secure Web Gateway (SWG) supports HTTP and HTTPS-based instant messaging protocols. As a result, when you use the Instant Messaging URL category to block messages, SWG can block messages to ICQ, for example, but cannot block messages from applications that use non-standard ports or tunneling over HTTP, such as, Yahoo Messenger, Skype, Google Talk, and so on.

Similarly, SWG cannot block messages from file-sharing and peer-to-peer protocols that do not use HTTP or HTTPS; most of these protocol types do not use either HTTP or HTTPS.

Adding custom URL categories to the URL database

Note: A URL database is available only on a BIG-IP® system with a Secure Web Gateway (SWG) subscription.
You can add a custom category to the standard Secure Web Gateway URL categories to specify a list of URLs that you want to block or allow, or for which you want to obtain confirmation from a user before blocking or allowing access.
Note: The URL categories that you add become subcategories of Custom Categories. Custom Categories take precedence over standard categories.
  1. On the Main tab, click Access Policy > Secure Web Gateway > URL Categories .
    The URL Categories table displays; Custom Categories displays as the first entry in the table.
  2. Click Create.
    The Category Properties screen displays.
  3. In the Name field, type a unique name for the URL category.
  4. From the Default Action list, retain the default value Block; or, select an alternative: Allow or Confirm.
    If no action has been specified in a filter for this category, the URL Filter agent takes the branch for the default action.
  5. Add, edit, or delete the URLs that are associated with the category by updating the Associated URLs list.
  6. To add URLs to the Associated URLs list:
    1. In the URL field, type a URL.
      You can type a well-formed URL that the system must match exactly or type a URL that includes globbing patterns (wildcards) for the system to match URLs.
    2. If you typed globbing patterns in the URL field, select the Glob Pattern Match check box .
    3. Click Add.
      The URL displays in the Associated URLs list.
    These are well-formed URLs:
    • https://www.siterequest.com/
    • http://www.siterequest.com:8080/
    • http://www.sitequest.com/docs/siterequest.pdf/
    • http://www.sitequest.com/products/application-guides/
    This URL *siterequest.[!comru] includes globbing patterns that match any URL that includes siterequest, except for siterequest.com or siterequest.ru.
    This URL *://siterequest.com/education/* includes globbing patterns that match any HTTP URL that includes siterequest.com/education, but that do not match any HTTPS URLs if Category Lookup specifies that the input is SNI or CN.Subject.
    Important: For SNI or CN.Subject input, Category Lookup uses scheme:://host for matching, instead of matching the whole URL.
  7. Click Finished.
    The URL Categories screen displays.
  8. To view the newly created URL category, expand Custom Categories.
    The custom URL category displays in the Sub-Category column.
Add or edit a URL filter to specify an action (allow, block, or confirm) for the custom category.

Customizing standard categories from the URL database

You can customize the standard URL categories supplied in the URL database by adding URLs to them. You might do this after you use APM as a forward proxy for a while, view logs and reports, and determine that you need to make changes.
Note: A URL database is available only on a BIG-IP® system with an SWG subscription.
Note: If you add a URL to a URL category, APM gives precedence to that categorization and database downloads do not overwrite your changes.
  1. On the Main tab, click Access Policy > Secure Web Gateway > URL Categories .
    The URL Categories table displays.
  2. Click the name of any category or subcategory to edit the properties for it.
    To view and select a subcategory, expand categories.
    The Category Properties screen displays. There are many URLs in a given category; however, any URLs that display on the Associated URLs list are entered by the user.
  3. Edit or delete any URLs on the Associated URLs list.
  4. To add URLs to the Associated URLs list:
    1. In the URL field, type a URL.
      You can type a well-formed URL that the system must match exactly or type a URL that includes globbing patterns (wildcards) for the system to match URLs.
    2. If you typed globbing patterns in the URL field, select the Glob Pattern Match check box .
    3. Click Add.
      The URL displays in the Associated URLs list.
    These are well-formed URLs:
    • https://www.siterequest.com/
    • http://www.siterequest.com:8080/
    • http://www.sitequest.com/docs/siterequest.pdf/
    • http://www.sitequest.com/products/application-guides/
    This URL *siterequest.[!comru] includes globbing patterns that match any URL that includes siterequest, except for siterequest.com or siterequest.ru.
    This URL *://siterequest.com/education/* includes globbing patterns that match any HTTP URL that includes siterequest.com/education, but that do not match any HTTPS URLs if Category Lookup specifies that the input is SNI or CN.Subject.
    Important: For SNI or CN.Subject input, Category Lookup uses scheme:://host for matching, instead of matching the whole URL.
  5. Click Add.
    The URL displays in the Associated URLs list.
  6. Click Update.
    The URL Properties screen refreshes.
  7. On the Main tab, click Access Policy > Secure Web Gateway > URL Categories .
    The URL Categories table displays. The screen displays (recategorized) next to the URL category that you customized.
URLs are added to the URL category that you selected.

Customizing URL filters for SWG

You configure a URL filter to specify whether to allow, block, or confirm requests for URLs in URL categories. You can configure multiple URL filters.
Note: On a BIG-IP® system with an SWG subscription, default URL filters, such as block-all and basic-security, are available. You cannot delete default URL filters.
  1. On the Main tab, click Access Policy > Secure Web Gateway > URL Filters .
    You can click the name of any filter to view its settings.
    The URL Filters screen displays.
  2. To configure a new URL filter, click one of these options.
    • Create button: Click to start with a URL filter that allows all categories.
    • Copy link: Click for an existing URL filter in the table to start with its settings.
  3. In the Name field, type a unique name for the URL filter.
  4. Click Finished.
    The screen redisplays. An Associated Categories table displays. It includes each URL category and the filtering action that is currently assigned to it. The table includes a Sub-Category column. Any URL categories that were added by administrators are subcategories within Custom Categories
  5. To block access to particular categories or subcategories, select them and click Block.
    Important: When you select a category, you also select the related subcategories. You can expand the category and clear any subcategory selections.
  6. Expand the category Miscellaneous, select Uncategorized, and then click Block.
    Important: It is important to block URLs that SWG cannot categorize.
  7. To allow access to particular categories or subcategories, select them and click Allow.
  8. To indicate that you want a user to confirm that access is work-related or otherwise justified before obtaining access to the URLs in a category, select the categories or subcategories and click Confirm.
To put a URL filter into effect, you must assign it in a per-request policy. A per-request policy runs each time a user makes a URL request.