You should consider using this procedure under the following condition:
- You want to capture a tcpdump that contains additional internal Traffic Management Microkernel (TMM) information.
TMM information can be helpful in advanced troubleshooting situations, such as tracking connection flows with multiple TMM instances on Clustered Multiprocessing (CMP) platforms.
You must meet the following prerequisites to use this procedure:
- You have command-line access to the BIG-IP system.
- You are familiar with the Wireshark third-party utility.
- You are familiar with interpreting tcpdump output files.
- You are familiar with the BIG-IP tcpdump utility.
Note: This article covers only the options for the tcpdump utility that are relevant to collecting internal TMM information. For general assistance with tcpdump, refer to the Supplemental Information section in this document.
The F5 implementation of the tcpdump utility can add internal TMM information to a tcpdump capture. In the course of a support case, an F5 Technical Support engineer may ask you to capture a tcpdump where this extra information is present. Or, you may want to collect the data yourself for analysis in a tool, such as Wireshark.
The enhanced tcpdump utility can capture extra details, such as which virtual server and which TMM is handling a specific sample of traffic. When reviewing the tcpdump output file in Wireshark, this extra information appears under the Ethernet II section in the Packet Details panel.
Note: The procedures in this article detail only how to collect the additional information using the tcpdump utility installed on the BIG-IP system. For more information about loading the tcpdump files and locating the packet details, refer to your Wireshark product manual.
Capturing extended TMM data with tcpdump
Impact of procedure: Refer to SOL6546: Recommended methods and limitations for running tcpdump on a BIG-IP system.
To capture internal TMM information, a noise amplitude operator is appended to the interface argument for a given tcpdump command, as shown in the following syntax:
tcpdump -i <interface>:<noise amplitude>
The noise amplitude defines the level of TMM details included in the packet capture. The following noise levels may be captured:
- n: Low details
- nn: Low and medium details
- nnn: Low, medium, and high details
F5 recommends always capturing the maximum noise level with the nnn option.
The noise levels include the following details:
- Ingress: A flag indicating whether TMM is sending or receiving the packet. A zero (0) indicates that TMM is sending the packet, while a non-zero number indicates that TMM is receiving the packet.
- Slot: The chassis slot number of the TMM that is handling the packet.
- TMM: The number of the TMM that is handling the packet.
- VIP: The name of the virtual server that is handling the connection. Prior to BIG-IP 11.2.0, the name was limited to 16 characters. In BIG-IP 11.2.0 and later, the name is limited to 96 characters.
- Flow ID: A number identifying a flow within TMM. The same flow ID can be used for different flows in different TMMs. Also, the same flow ID can be re-used for a different flow within the same TMM at a different time.
- Peer ID: A number identifying the peer flow within TMM. Note that the same peer ID can be used for different flows in different TMMs. Also, the same peer ID can be re-used for a different flow within the same TMM at a different time.
- Reset Cause: In BIG-IP 11.2.0 and later, the reset cause (if available) is included for TCP reset packets. For more information, refer to SOL13223: Configuring the BIG-IP system to log TCP RST packets.
- Connflow Flags: Diagnostic information used by F5 Technical Support.
- Flow Type: Diagnostic information used by F5 Technical Support.
- High Availability Unit: Diagnostic information used by F5 Technical Support.
- Ingress Slot: Diagnostic information used by F5 Technical Support.
- Ingress Port: Diagnostic information used by F5 Technical Support.
- Peer IP Protocol: The IP protocol of the peer flow. This field is not populated prior to BIG-IP 11.0.0.
- Peer VLAN: The VLAN ID number that is associated with the peer flow.
- Peer Remote Address: The IP address of the host on the far end of the peer flow.
- Peer Local Address: The IP address used by TMM for the peer flow.
- Peer Remote Port: The protocol port of the host on the far end of the peer flow.
- Peer Local Port: The protocol port used by TMM for the peer flow.
- The extra TMM information can only be captured when the interface on which tcpdump is listening is a Virtual Local Area Network (VLAN). The extra information is not included in the dump if tcpdump is listening on a physical interface (1.1 or 2.2), which would display the traffic ingressing or egressing the BIG-IP system through the interface before or after TMM has processed it.
- The -s0 option must be specified to cause tcpdump to capture the whole packet and extra TMM information.
- Some of the fields are not always populated. For example, when the first packet arrives for a connection, the VIP and flow information is unavailable.
- Some of the fields above may exist in different detail levels between different software releases.
- The term, "peer" is used to describe and identify a connection. A connection is two flows. Each flow is a peer of the other. For example:
Packet from client to BIG-IP 10.1.1.1:1234 -> 10.1.1.3:80
flow id: 5678
peer id: 4356
Peer remote address: 10.2.1.5
Peer remote port: 80
Peer local address: 10.2.1.3
Peer local port: 1234
Packet from server to BIG-IP 10.2.1.3:1234-> 10.2.1.5:80
Flow id: 4356
Peer id: 5678
Peer remote address: 10.1.1.1
Peer remote port: 1234
Peer local address: 10.1.1.3
Peer local port: 80
Capturing traffic with TMM information for use with Wireshark
- To view the extended TMM information, the tcpdump capture must first be output to a file with the -w option. With the addition of the options detailed in the previous sections, a complete command to capture extended TMM data in a tcpdump would use the following syntax:
tcpdump -s0 -ni <vlan>:<noiseamplitude> -w <path to output file> <filter options>
tcpdump -s0 -ni internal:nnn -w /var/tmp/my_output_file.dmp
- The standard version of the Wireshark program does not display TMM information. F5 Product Development has created a Wireshark plug-in that can decode the additional TMM information encoded in a tcpdump capture. To obtain the required plug-in, download the F5 Wireshark plug-in from the F5 Wireshark Plugin page on DevCentral.
Note: A DevCentral login is required to access this content.
- The plugin can also be used to filter the dump using some of the additional F5 details. For example, the following Wireshark filter string shows traffic to and from TMM0 on Slot1:
f5ethtrailer.slot == 1 and f5ethtrailer.tmm == 0
A list of all F5 filters is shown in Wireshark within the Filter Expression window.
Capturing traffic with TMM information for a specific traffic flow
Beginning in BIG-IP 11.2.0, you can use the 'p' interface modifier with the 'n' modifier to capture traffic with TMM information for a specific flow, and its related peer flow. The p modifier allows you to capture a specific traffic flow through the BIG-IP system from end to end, even when the configuration uses a Secure Network Address Translation (SNAT) or OneConnect. For example, the following command searches for traffic to or from client 10.0.0.1 on interface 0.0:
tcpdump -ni 0.0:nnnp -s0 -c 100000 -w /var/tmp/capture.dmp host 10.0.0.1
Once tcpdump identifies a related flow, the flow is marked in TMM, and every subsequent packet in the flow (on both sides of the BIG-IP system) is written to the capture file.