When you install a Global Traffic Manager system in a network, that
system typically works within a larger group of BIG-IP®
products. These products include other Global Traffic Manager systems, Link Controller systems, and Local Traffic Manager systems. Global Traffic Manager must be able to communicate with these other systems to maintain an accurate assessment of the health and availability of different network components. For example, Global Traffic Manager must be able to acquire statistical data from resources that are managed by Local Traffic Manager in a different data center. BIG-IP systems acquire this information through the use of probes. A probe
is an action a BIG-IP system takes to acquire data from other network resources.
Probes are an essential means by which Global Traffic Manager tracks the
health and availability of network resources; however, it is equally important that the responsibility for conducting probes be distributed across as many BIG-IP products as possible. This distribution ensures that no one system becomes overloaded with conducting probes, which can cause a decrease in performance in the other tasks for which a BIG-IP system is responsible.
To distribute probe requests effectively across multiple BIG-IP systems,
Global Traffic Manager systems employ several different technologies and methodologies, including:
| || |iQuery®
, which is the communication protocol used between Global Traffic Manager systems and the big3d
agents that reside on other BIG-IP systems
One of the important concepts to remember when understanding how Global
Traffic Manager acquires network data is that the process consists of several tasks:
| || |The big3d
agent conducts the probe.
| || |The big3d
agent broadcasts the results of the probe, allowing all Global Traffic Manager systems to receive the information.
At the heart of probe management with Global Traffic Manager systems is
iQuery, the communications protocol that these systems use to send information from one system to another. With iQuery, Global Traffic Manager systems in the same synchronization group can share configuration settings, assign probe requests to big3d
agents, and receive data on the status of network resources.
The iQuery protocol is an XML protocol that is sent between each system
using gzip compression and SSL. These communications can only be allowed between systems that have a trusted relationship established, which is why configuration tools such as big3d_install
, and gtm_add
are critical when installing or updating Global Traffic Manager systems. If two systems have not exchanged their SSL certificates, they cannot share information with each other using iQuery.
In addition to requiring trusted relationships, systems send iQuery
communications only on the VLAN on which the system received the incoming message. Also, iQuery communications occur only within the same synchronization group. If your network consists of two synchronization groups, with each group sharing a subset of network resources, these groups probe the network resources and communicate with iQuery separately.
Generally, iQuery communications require no user intervention; however,
on occasion it can be necessary to view the data transmitted between each system. For example, you might be troubleshooting the reason that a Global Traffic Manager system is exhibiting a particular behavior. In such a situation, you can use the command, iqdump
When you assign a monitor to a network resource, Global Traffic Manager
is responsible for ensuring that a big3d
agent probes the selected resource. It is important to remember that this does not necessarily mean the selected Global Traffic Manager actually conducts the probe; it means only that a specific Global Traffic Manager is in charge of assigning a big3d
agent to probe the resource. The big3d
agent can be installed on the same system as Global Traffic Manager, a different Global Traffic Manager, or another BIG-IP system.
A crucial component to determining which system manages a probe request
is the data centers that you define in the Global Traffic Manager configuration. For each probe, the Global Traffic Manager systems determine the following:
By default, Global Traffic Manager systems delegate probe management to
a system that belongs to the same data center as the resource, since the close proximity of system and resource improves probe response time.
To illustrate how these considerations factor into probe management,
consider a fictional company, SiteRequest. This company has three data centers: one in Los Angeles, one in New York, and one in London. The following table lists a few characteristics of each data center.
Now, consider that you want to acquire statistical data from a resource in the
New York data center. First, the Global Traffic Manager systems, based on their iQuery communications with each other, identify whether there is a Global Traffic Manager system that belongs to the New York data center. In this case, the answer is yes; the New York data center contains a Global Traffic Manager system. Next, the systems determine if more than one Global Traffic Manager belongs to the New York data center. In this case, the answer is no; the New York data center has only a stand-alone system. Consequently, the Global Traffic Manager system in the New York data center assumes responsibility for conducting the probe on this particular resource.
In situations where more than one Global Traffic Manager belongs to a data
center, the systems use an algorithm to distribute the responsibility for probes equally among Global Traffic Manager systems. This distribution ensures that each Global Traffic Manager has an equal chance of being responsible for managing a probe request.
To demonstrate how probe requests are delegated between two Global
Traffic Manager systems at the same data center, consider again the network configuration at SiteRequest. This time, the company needs to acquire data from a resource that resides at the Los Angeles data center. As with the previous example, the first step identifies whether the Los Angeles data center has any Global Traffic Manager systems; in this case, the answer is yes. The next criteria is whether there is more than one Global Traffic Manager at that data center; in this case, the answer is also yes: the Los Angeles data center has a redundant system configuration that consists of two Global Traffic Manager systems. Because there are two Global Traffic Manager systems at this data center, each system compares the hash value of the resource with its own information; whichever Global Traffic Manager has the closest value to the resource becomes responsible for managing the probe request.
A final consideration is if a data center does not have any Global Traffic
Manager systems at all, such as the London data center in the configuration for SiteRequest. In this situation, the responsibility for probing a resource at that data center is divided among the other Global Traffic Manager systems; much in the same way as the responsibility is divided among Global Traffic Manager systems within the same data center.
When Global Traffic Manager becomes responsible for managing a probe, it
remains responsible for that probe until the network configuration changes in one of the following ways:
The first stage in conducting a probe of a network resource is to select the
Global Traffic Manager system. In turn, Global Traffic Manager delegates the probe to a big3d
agent, which is responsible for querying the given network resource for data.
The probe delegation of network resources process is similar to the
two-tiered load balancing method Global Traffic Manager uses when delegating traffic. With DNS traffic, Global Traffic Manager identifies the wide IP to which the traffic belongs, and then load balances that traffic among the pools associated with the wide IP. After it selects a pool, the system load balances the request across the pool members within that pool.
Delegating probe requests occurs in a similar two-tiered fashion. First, the
Global Traffic Manager systems within a synchronization group determine which system is responsible for managing the probe. This does not necessarily mean that the selected Global Traffic Manager conducts the probe itself; it means only that a specific Global Traffic Manager ensures that the probe takes place. Next, Global Traffic Manager selects one of the available big3d
agents to actually conduct the probe. As each BIG-IP system has a big3d
agent, the number of agents available to conduct the probe depends on the number of BIG-IP systems.
To illustrate how these considerations factor into probe management,
consider the fictional company, SiteRequest. This company has three data centers: one in Los Angeles, one in New York, and one in London. The following table lists a few characteristics of each data center:
Consider that a Global Traffic Manager system in the Los Angeles data
center has assumed responsibility for managing a probe for a network resource. At this data center, the system can assign the probe to one of four big3d
agents: one for each BIG-IP system at the data center. To select a big3d
, Global Traffic Manager looks to see which big3d
agent has the fewest number of probes for which it is responsible. The big3d
agent with the lowest number of probes is tasked with conducting the probe. Global Traffic Manager checks this statistic each time it needs to delegate the probe; as a result, the selected big3d
can change from probe instance to probe instance.
In situations where a big3d
agent does not reside in the same data center as the resource, the designated Global Traffic Manager selects a big3d
from all available big3d
agents on the network. Again, the agent selected is the agent with the fewest number of probe requests, and this check occurs each time the probe is conducted.
For example, SiteRequest adds a new set of web servers in Tokyo. At this
location, the company has yet to install its BIG-IP systems; however, the current set of Global Traffic Manager systems in Los Angeles and New York are managing traffic to these web servers. When initiating a probe request to determine the availability of one of these servers, a Global Traffic Manager system is selected to manage the probe request. Then, that system chooses a big3d
agent to probe the web server, selecting any big3d
agent located in Los Angeles, New York, or London.
Global Traffic Manager systems are responsible for probes of local DNS
servers (LDNS). Unlike probes conducted on internal systems, such as web servers, probes of local DNS servers require that the Global Traffic Manager system verifies data from a resource that exists outside the network. Typically, this data is the path information Global Traffic Manager requires when conducting Quality of Service, Round Trip Time, Completion Rate, and Hops load balancing methods.
When a given LDNS makes a DNS request for a wide IP, that request is sent
to a single Global Traffic Manager. Global Traffic Manager then creates an LDNS entry, and assigns that entry one of the following states:
| || |New
: Global Traffic Manager has not come across this particular LDNS before
| || |Active
: Global Traffic Manager already has an existing entry for this LDNS
| || |Pending
: Global Traffic Manager has been contacted by this LDNS before, however, this server has yet to respond to a probe from a Global Traffic Manager system on this network
In general, the New and Pending states are temporary states; an LDNS
remains in one of these states only until it responds to the first probe request from Global Traffic Manager. After Global Traffic Manager receives a response, the LDNS entry is moved to the Active state. Each Global Traffic Manager within a given synchronization group shares the LDNS entries that are assigned this state, resulting in the synchronization group having a common list of known local DNS servers.
Unlike internal probes, LDNS probes are not load balanced across Global
Traffic Manager systems. Instead, the Global Traffic Manager system that the LDNS first queries becomes responsible for the initial probe to that LDNS. These probes are load balanced, however, across the multiple big3d
agents, with preference given to big3d
agents that either belong to the same data center as the responding Global Traffic Manager, or belong to the same link through which Global Traffic Manager received the LDNS query. After the initial probe, an algorithm is used to load balance subsequent probes across the available Global Traffic Manager systems.
Probes are the means by which Global Traffic Manager tracks the health and
availability of network resources, and it is important that the responsibility for conducting probes is distributed across as many BIG-IP products as possible. You can use information in the Global Traffic Manager log file to determine how to fine tune the probes that you have configured. However, the probe logs feature is disabled by default. You must turn on the feature for the probe information to appear in the log file.
If you want Global Traffic Manager to gather information about probes and
save it in the log file, you must set the database variable GTM.DebugProbeTuningInterval
to a non-zero value. The value of the variable indicates, in seconds, how often you want the system to add probe information to the log file. By default this variable is set to 0
(zero), which disables the logging of information about probes.
The probe information displays in the logs in the Configuration utility when
setting on the Logs screen is set to the default value of Notice
. When you set the GTM.DebugProbeTuningInterval
database variable to a non-zero value, the log file contains information about probes including the number of local DNS servers, Global Traffic Manager systems, paths, and persistence records in your network. The log file also includes the information in the following list.
| || |Number of up
instances and the average and maximum probe time for each up
| || |Number of down
instances, the average probe time for each down
instance, and a sorted list of reasons that the instance is down. Each reason in the list is followed the number of instances that were marked down
for this reason.
| || |Memory usageNote: This value is -1, unless an SNMP monitor is assigned to the server.