NSX-T Packet Capture: East - West Traffic on Overlay Segment

In this article, we are going to perform packet captures at different capture points between 2 VMs running on 2 Overlay segments attached to a T1 Router.

Topology

The Tier-1 Gateway does not provide stateful services so it's not connected to an Edge Cluster. It worth mentioning it because the data path will only go from ESXi host to ESXi host without the need of traversing the physical network.

Let's to a Traceflow and see what the data path looks like:

The traffic goes from the VM vnic down to the vdr port

We need to get the PortIDs for:

• T1-Web-01
• T1-DB-01
• vdr port on the source host (Where the routing decisions take place)

We can use many different methods to find this information, but i thhink the most efficient is nsxdb-cli

1. Get an SSH session to the host where the source VM (T1-Web-01) is running and do: nsxdp-cli vswitch instance list

   

2. Now do the same on the second host:

   

We are interested in:

Ok, now let's start a continuous ping from T1-Web-01 to T1-DB-01:

And start the capture at the switchport level:

1# pktcap-uw --switchport 100663322 --dir 2 -o - | tcpdump-uw -ner -

The packet is sent from the VM vnic 00:50:56:92:49:09 to the vdrport 02:50:56:56:44:52. ICMP replies comes from the same MAC.

On the other side T1-DB-01 the very same thing is happening. ICMP packets coming from 02:50:56:56:44:52 are delivered to 00:50:56:92:98:b6 which replies to the same:

Now let's dig a little bit deeper and follow the data path and capture at different capture points:

Capturing traffic using nsxcli

From NSX-T Command-Line Interface Reference

Sintax: start capture interface <interface-name> [direction <direction>] [file <filename>] [count <packet-count>] [expression <expression>]

Point	Description	Capture with nsxcli
1	VM vnic switchport	nsxcli -c start capture interface 100663322 direction dual expression ipproto 0x01
2	Before the DFW	nsxcli -c start capture dvfilter nic-534759-eth0-vmware-sfw.2 stage pre expression ipproto 0x01
3	After the DFW	nsxcli -c start capture dvfilter nic-534759-eth0-vmware-sfw.2 stage post expression ipproto 0x01
4	At the vdr Port	nsxcli -c start capture interface vdrPort expression ipproto 0x01 Source host (requests only): Destination host (replies only):
5	N-DVS Uplink	nsxcli -c start capture interface c9b5052b-9e46-4c9e-9b2a-8b1b46e26796 direction dual expression ipproto 0x01 To get the DVPortID:
6	Physical Uplink	nsxcli -c start capture interface vmnic2 direction dual Here the traffic is encapsulated into GENEVE protocol
7	Before the DFW	To get the dvfilter name: nsxcli -c start capture dvfilter nic-534759-eth0-vmware-sfw.2 stage pre expression ipproto 0x01
8	After the DFW	nsxcli -c start capture dvfilter nic-7925274-eth0-vmware-sfw.2 stage post expression ipproto 0x01
9	VM vnic switchport	nsxcli -c start capture interface 134217801 direction dual expression ipproto 0x01

Capturing traffic using pktcap-uw