Applies To:

Show Versions Show Versions

Manual Chapter: Writing Custom Bot Signatures
Manual Chapter
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Writing Custom Bot Signatures

About bot signatures

Bot signatures identify web robots by looking for specific patterns in the headers of incoming HTTP requests. DoS Layer 7 bot detection includes many signatures that identify bots, and you can also write your own for customized bot defense.

Bot signatures carefully identify bots and have a low rate of producing false positive results. The signatures identify the type of bot for classification and investigative purposes, and can distinguish between benign and malicious bots.

Benign bots can be useful for providing Internet services such as search engine bots, index crawlers, site monitors, and those used to establish availability and response time. Some environments may not want to block benign bot traffic. But attackers use malicious bots for more harmful purposes such as harvesting email addresses, producing spam, and developing exploitation tools. You may want to block malicious bots because they can orchestrate DoS attacks, waste internet resources, and search for vulnerabilities to exploit in your application.

Being able to classify bots allows you to treat them differently. You can report, block, or do nothing when a signature matches a malicious or benign bot. Further, malicious and benign bots fall into more specific bot signature categories that can be handled as needed. You can create new categories if needed for custom bot signatures.

Bot signature syntax limitations

Bot signatures are developed using Snort syntax to search for bots in either the User-Agent field of the header or the URL, or both. The User-Agent field is examined to identify the browser and operating system. The URL is searched to locate bots that access specific peculiar URLs within a site, regardless of whether the site has such a URL (in most cases it does not).

The syntax of bot signatures is similar to that of attack signatures using the general format keyword: "value"; modifier; but bot signatures can include only the following attributes:

  • uricontent (objonly flag is not allowed)
  • headercontent (useragentonly flag is mandatory)
  • ipp (for regular expressions)
  • offset
  • depth
  • nocase

The following are not allowed in bot signatures:

  • negation
  • norm (normalization is predefined)
  • distance modifiers
  • within modifiers

Refer to the Signature Options, Signature Syntax, and examples for additional details on the syntax used in bot signatures.

Creating a custom bot signature

You can write custom bot signatures to increase bot protection for your web application. Bot signatures are written similar to attack signatures using a limited subset of allowed keywords. You can design custom bot signatures to handle security policy enforcement unique to your networking environment, emergency situations, or to provide an analysis of specific activity on the network. If your organization needs a custom bot signature, you can create one.
  1. On the Main tab, click Security > Options > DoS Protection > Bot Signatures > Bot Signatures List .
  2. Click Create.
    The Create New Bot Signature screen opens.
  3. In the Name field, type a unique name for the bot signature.
    Note: If you attempt to create a custom bot signature with the same name as a system-supplied signature, you will receive an error and the signature will not be created.
  4. In the Domains field, type the name of one or more domains from which the bot can send its requests, and click Add.
    This field is mandatory only for benign bots in the Search Engines category to validate their identity using reverse DNS lookup.
  5. From the Category list, select the type of bot this signature will protect against.
    All bot categories are listed including those provided by the system and any that you have created.
  6. In the Rule setting, develop the bot signature.
    1. If the bot signature will search for patterns in the user-agent string or URL, use Simple Edit Mode.
    2. If searching the user-agent string, for User-agent, select either contains or regexp match, and then type the pattern in the text box.
    3. If searching requests within the normalized URI path (excluding the Query String), for URL, select either contains or regexp match, and then type the pattern in the text box.
    Note: If a bot signature requires multiple search strings, a conditional text match, or a search of both the URL and User-agent contexts, you cannot use the simple edit mode.

    To develop more complex bot signatures, use the Advanced Edit Mode to type expressions using Snort control. Refer to the Signature Options and Signature Syntax sections for details. Refer to Bot signature syntax for special limitations when writing bot signatures.

    As an example, this signature searches the header for three terms: FOO, BAR, and for a specific hexadecimal value.

    headercontent:\"FOO\"; useragentonly; nocase;
    headercontent:\"BAR\"; useragentonly; nocase; 
    headercontent:\"0x31303235343830303522\"; useragentonly; nocase;

    In this example, the bot signature searches both User-Agent and the URL:

    headercontent: "MaliciousBot/0.1"; useragentonly; 
    uricontent: "/settings.php";
  7. For the Risk setting, select a risk level.
    The risk level indicates the level of potential damage an attack containing this text may cause, if it were successful.
  8. Click Create to create the new bot signature.
The new bot signature is added to the bot signature list. If you create a DoS profile, enable Bot Signature Check, and associate the DoS profile with a virtual server, the system examines traffic for bots (using the signature you developed plus others that are enabled) as one of the first checks.

Creating a new bot signature category

You can create customized bot signature categories if the default categories do not include the one you are looking for. You can create a new malicious or benign bot category.
  1. On the Main tab, click Security > Options > DoS Protection > Bot Signature Categories .
    The Bot Signature Categories screen opens.
  2. Depending on whether you want to create a malicious or benign category, in the edit field above the Malicious Categories List or Benign Categories List, type a unique name for the category and click Add.
  3. Click Save to save the new bot signature category on the system.
The bot signature category is added to the appropriate list. If you create a customized bot signature, you can assign the category you created to that signature.
Table of Contents   |   << Previous Chapter   |   Next Chapter >>

Was this resource helpful in solving your issue?

NOTE: Please do not provide personal information.

Additional Comments (optional)