다음과 같은 구성 환경에서 NAT의 Tier-1 Gateway에 NAT 설정을 하는 경우, Source <-> Target간 Traffic이 어떻게 흐르는지에 대해서 알아봅니다.
[구성 환경]
SNAT/DNAT 설정
Edge에서 설정 조회
edge-node-01> get firewall c6354b9d-9749-4b03-b989-672c3dd497b8 ruleset rules ### <-- !! Tier-1의 Uplink Interface Thu Sep 14 2023 UTC 07:01:32.136 DNAT rule count: 1 Rule ID : 536870912 Rule : in protocol any postnat from any to ip 10.31.1.2 dnat ip 172.31.1.2 SNAT rule count: 1 Rule ID : 536870913 Rule : out protocol any prenat from ip 172.31.1.2 to any snat ip 10.31.1.2 Firewall rule count: 1 Rule ID : 1002 Rule : inout protocol any from any to any accept edge-node-01> get firewall ed6f81d5-d520-4fd6-9bce-1fb980b76ff7 ruleset rules ### <-- !! Tier-1의 Downlink Interface Thu Sep 14 2023 UTC 07:02:16.795 DNAT rule count: 1 Rule ID : 536870912 Rule : in protocol any postnat from any to ip 10.31.1.2 dnat ip 172.31.1.2 SNAT rule count: 1 Rule ID : 536870913 Rule : out protocol any prenat from ip 172.31.1.2 to any snat ip 10.31.1.2 Firewall rule count: 0 |
NAT Connection State 확인
1. 192.168.1.2 → 10.31.1.2 IP로 ICMP 테스트
C:\>ping 10.31.1.2 -t Pinging 10.31.1.2 with 32 bytes of data: Reply from 10.31.1.2: bytes=32 time=1ms TTL=125 Reply from 10.31.1.2: bytes=32 time=1ms TTL=125 |
2. Tier-1 Gateway Interface 확인 : Tier-1의 Uplink와 Downlink 확인
edge-node-01> get firewall interfaces Thu Sep 14 2023 UTC 02:34:19.653 Interface : b293703d-622b-4cd7-9173-65efb2f07bd4 Type : UPLINK Sync enabled : true Name : edge01-uplink01 VRF ID : 1 Context entity : ec318044-8bd6-4269-aa47-544ee747f765 Context name : SR-tier0-01 Interface : 26b8d93c-bdd1-4155-b31f-5bf966009615 Type : BACKPLANE Sync enabled : true Name : bp-sr0-port VRF ID : 1 Context entity : ec318044-8bd6-4269-aa47-544ee747f765 Context name : SR-tier0-01 Interface : aeb063b7-c37c-43dd-a43f-44f97d92dabb Type : UPLINK Sync enabled : true Name : edge01-uplink02 VRF ID : 1 Context entity : ec318044-8bd6-4269-aa47-544ee747f765 Context name : SR-tier0-01 Interface : 3f340ce8-eecb-4de6-b7fc-66f1a03ae599 Type : BACKPLANE Sync enabled : true Name : bp-sr0-port VRF ID : 4 Context entity : 064b8812-743c-427a-b0b6-801570118070 Context name : SR-tier1-01 Interface : c6354b9d-9749-4b03-b989-672c3dd497b8 ### <-- !! Type : UPLINK Sync enabled : true Name : tier0-01-tier1-01-t1_lrp VRF ID : 4 Context entity : 064b8812-743c-427a-b0b6-801570118070 Context name : SR-tier1-01 Interface : f6534080-6e5a-5f4a-9071-0758da2af81b Type : DOWNLINK Sync enabled : true Name : tier0-01-tier1-01-t0_lrp VRF ID : 3 Context entity : 73bdcf75-0251-4927-9c80-68a6d7265911 Context name : DR-tier0-01 Interface : ed6f81d5-d520-4fd6-9bce-1fb980b76ff7 ### <-- !! Type : DOWNLINK Sync enabled : true Name : infra-overlay-seg-3101-dlrp VRF ID : 6 Context entity : 92045476-c754-48e1-a86a-1ac4f9961112 Context name : DR-tier1-01 |
3. 위에서 확인한 Firewall Interface를 이용하여 Connection State 조회
edge-node-01> get firewall c6354b9d-9749-4b03-b989-672c3dd497b8 connection state Thu Sep 14 2023 UTC 02:35:50.635 Connection count: 1 192.168.1.2 -> 172.31.1.2 (10.31.1.2) dir in protocol icmp f-1002 n-536870912 edge-node-01> get firewall ed6f81d5-d520-4fd6-9bce-1fb980b76ff7 connection state Thu Sep 14 2023 UTC 02:36:18.325 Connection count: 0 |
Packet 확인
1. Tier-1 Uplink에서 Packet Capture : 192.168.1.2 → 10.31.1.2 간 Packet 확인
edge-node-01> start capture interface c6354b9d-9749-4b03-b989-672c3dd497b8 expression icmp and host 192.168.1.2 02:47:43.575618 02:50:56:56:44:52 > 02:50:56:56:44:55, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 10.31.1.2: ICMP echo request, id 1, seq 31630, length 40 <base64>AlBWVkRVAlBWVkRSCABFAAA8r2YAAH4BwI/AqAECCh8BAggA0cwAAXuOYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> 02:47:43.576420 02:50:56:56:44:55 > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 10.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 31630, length 40 <base64>AlBWVkRSAlBWVkRVCABFAAA8gaUAAH8B7VAKHwECwKgBAgAA2cwAAXuOYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
2. Tier-1 Downlink에서 Packet Capture : 192.168.1.2 → 172.31.1.2 간 Packet 확인
edge-node-01> start capture interface ed6f81d5-d520-4fd6-9bce-1fb980b76ff7 08:02:02.408033 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 49976, length 40 <base64>AFBWoaTvAlBWVkRSCABFAAA89ukAAH0B2AvAqAECrB8BAggAiiIAAcM4YWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> 08:02:03.423992 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 49977, length 40 <base64>AFBWoaTvAlBWVkRSCABFAAA89uoAAH0B2ArAqAECrB8BAggAiiEAAcM5YWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
3. Tier-1 SR의 backplance Interface에서 ICMP Reply Packet 확인
edge-node-01> start capture interface 3f340ce8-eecb-4de6-b7fc-66f1a03ae599 07:18:13.414900 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 47367, length 40 <base64>AlBWVlMAAlBWVkRSCABFAAA8vw8AAH8BDeasHwECwKgBAgAAnFMAAbkHYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> 07:18:14.429101 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 47368, length 40 <base64>AlBWVlMAAlBWVkRSCABFAAA8vxAAAH8BDeWsHwECwKgBAgAAnFIAAbkIYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
4. vNIC에서 Packet Capture
[root@esxi701:~] nsxdp-cli vswitch instance list DvsPortset-0 (DSwitch-nsx-overlay) 50 21 a0 e8 34 5c 98 8d-82 40 a5 d4 44 76 96 b0 Total Ports:4850 Available:4827 Client PortID DVPortID MAC Uplink VID VNI Management 67108871 00:00:00:00:00:00 n/a N/A N/A vmnic4 2214592528 5 00:00:00:00:00:00 0-4094 N/A Shadow of vmnic4 67108881 00:50:56:52:85:fe n/a N/A N/A vmnic3 2214592530 4 00:00:00:00:00:00 0-4094 N/A Shadow of vmnic3 67108883 00:50:56:55:90:16 n/a N/A N/A vmk10 67108887 315bf459-288b-4008-96af-77582fa10218 00:50:56:64:8e:0f vmnic3 0 N/A vmk11 67108888 3026e43e-5751-4211-a058-ef0e250c1b94 00:50:56:60:2d:68 vmnic4 0 N/A vmk50 67108889 32ccc9d5-211a-4c71-8273-230e3dc9dc83 00:50:56:63:d0:49 void 1073742824 vdr-vdrPort 67108890 vdrPort 02:50:56:56:44:52 vmnic4 N/A overlay-test-vm.eth0 67108894 4100fda9-02a7-43a8-a76a-acaf00cbd764 00:50:56:a1:a4:ef vmnic4 0 67585 [root@esxi701:~] pktcap-uw --switchport 67108890 --dir 2 --proto 0x1 -o - | tcpdump-uw -r - -nne The switch port id is 0x0400001a. The session filter IP protocol is 0x1. pktcap: The output file is -. pktcap: No server port specifed, select 39108 as the port. pktcap: Local CID 2. pktcap: Listen on port 39108. pktcap: Main thread: 502721719168. pktcap: Dump Thread: 502722254592. pktcap: Recv Thread: 502722782976. pktcap: Accept... pktcap: Vsock connection from port 1034 cid 2. reading from file -, link-type EN10MB (Ethernet) 04:46:54.275198 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38715, length 40 04:46:54.275219 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38715, length 40 04:46:55.285542 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38716, length 40 04:46:55.285560 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38716, length 40 04:46:56.296683 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38717, length 40 04:46:56.296702 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38717, length 40 04:46:57.307235 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38718, length 40 tcpdump-uw: pcap_loop: error reading dump file: Interrupted system call pktcap: Join with dump thread failed. pktcap: Destroying session 8. pktcap: pktcap: Dumped 7 packet to file -, dropped 0 packets. pktcap: Done. [root@esxi701:~] pktcap-uw --switchport 67108894 --dir 2 --proto 0x1 -o - | tcpdump-uw -r - -nne The switch port id is 0x0400001e. The session filter IP protocol is 0x1. pktcap: The output file is -. pktcap: No server port specifed, select 40031 as the port. pktcap: Local CID 2. pktcap: Listen on port 40031. pktcap: Main thread: 660053183360. pktcap: Dump Thread: 660053718784. pktcap: Recv Thread: 660054247168. pktcap: Accept... reading from file -, link-type EN10MB (Ethernet) pktcap: Vsock connection from port 1035 cid 2. 04:48:10.183869 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 38790, length 40 04:48:10.183880 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 38790, length 40 04:48:10.184061 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38790, length 40 04:48:11.194934 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 38791, length 40 04:48:11.194934 02:50:56:56:44:52 > 00:50:56:a1:a4:ef, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.2: ICMP echo request, id 1, seq 38791, length 40 04:48:11.195147 00:50:56:a1:a4:ef > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 38791, length 40 tcpdump-uw: pcap_loop: error reading dump file: Interrupted system call pktcap: Join with dump thread failed. pktcap: Destroying session 9. pktcap: pktcap: Dumped 6 packet to file -, dropped 0 packets. pktcap: Done. [root@esxi701:~] nsxcli -c start capture interface vdrPort expression ipproto 0x01 04:59:14.854837 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39449, length 40 04:59:15.865288 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39450, length 40 04:59:16.876563 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39451, length 40 04:59:17.889121 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39452, length 40 04:59:18.900383 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39453, length 40 04:59:19.912558 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39454, length 40 04:59:20.923302 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39455, length 40 04:59:21.934693 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39456, length 40 04:59:22.945673 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39457, length 40 04:59:23.956270 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39458, length 40 04:59:24.967004 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39459, length 40 04:59:25.977146 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39460, length 40 04:59:26.987946 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39461, length 40 04:59:27.999791 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39462, length 40 04:59:29.010570 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.2 > 192.168.1.2: ICMP echo reply, id 1, seq 39463, length 40 Thu Sep 14 2023 UTC 04:59:14.061 % Stopped by Ctrl-C |
VRF 정보 확인
edge-node-01> get logical-routers Thu Sep 14 2023 UTC 04:36:29.261 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 6/5000 ec318044-8bd6-4269-aa47-544ee747f765 1 2 SR-tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 73bdcf75-0251-4927-9c80-68a6d7265911 3 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 5 2/50000 064b8812-743c-427a-b0b6-801570118070 4 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 92045476-c754-48e1-a86a-1ac4f9961112 6 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 1/50000 edge-node-01> vrf 064b8812-743c-427a-b0b6-801570118070 edge-node-01(tier1_sr[4])> get interfaces Thu Sep 14 2023 UTC 04:36:41.351 Logical Router UUID VRF LR-ID Name Type 064b8812-743c-427a-b0b6-801570118070 4 9 SR-tier1-01 SERVICE_ROUTER_TIER1 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : c6354b9d-9749-4b03-b989-672c3dd497b8 Ifuid : 294 Name : tier0-01-tier1-01-t1_lrp Fwd-mode : IPV4_ONLY Mode : lif Port-type : uplink IP/Mask : 100.64.0.1/31;fe80::50:56ff:fe56:4455/64(NA);fc14:9b49:677c:6c00::2/64(NA) MAC : 02:50:56:56:44:55 VNI : 74753 Access-VLAN : untagged LS port : 57ce7468-fdfd-4c36-824a-fae3ba5df6b5 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : 677cd5b2-09a1-4f00-b3a1-354081b7aad7 Ifuid : 292 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : 3f340ce8-eecb-4de6-b7fc-66f1a03ae599 Ifuid : 291 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/28;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 66560 Access-VLAN : untagged LS port : af64ecbb-fb0e-4e0a-b08b-e112dfa283f8 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : fa2373fe-c7d2-5c24-841b-09ae7a48830f Ifuid : 289 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 7cd6efc0-5ea7-5e0f-b83e-d6808c5bc745 Ifuid : 290 Mode : blackhole Port-type : blackhole Logical Router UUID VRF LR-ID Name Type 92045476-c754-48e1-a86a-1ac4f9961112 6 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : e674a6ec-ba06-4442-b303-81c5da4ae6f2 Ifuid : 310 Name : bp-dr-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.1/28;fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 ### <-- !! VNI : 66560 Access-VLAN : untagged LS port : ec5c44bd-a85f-4f85-9791-227fba216782 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : ed6f81d5-d520-4fd6-9bce-1fb980b76ff7 Ifuid : 307 Name : infra-overlay-seg-3101-dlrp Fwd-mode : IPV4_ONLY Mode : lif Port-type : downlink IP/Mask : 172.31.1.1/24 MAC : 02:50:56:56:44:52 ### <-- !! VNI : 67585 Access-VLAN : untagged LS port : fe79606b-0572-428b-b35a-63cf922c931b Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : a69fbf2f-f4c6-5da9-9479-ca465099c846 Ifuid : 304 Mode : cpu Port-type : cpu Enable-mcast : false Interface : a6281bc0-8795-5b05-a774-f43910cc4daa Ifuid : 305 Mode : blackhole Port-type : blackhole |
Flow 정리
ICMP Request
c6354b9d-9749-4b03-b989-672c3dd497b8(tier0-01-tier1-01-t1_lrp/uplink) -> ed6f81d5-d520-4fd6-9bce-1fb980b76ff7(infra-overlay-seg-3101-dlrp/downlink)
192.168.1.2 > 10.31.1.2 192.168.1.2 > 172.31.1.2
02:50:56:56:44:52 > 02:50:56:56:44:55 02:50:56:56:44:52 > 00:50:56:a1:a4:ef
ICMP Reply
3f340ce8-eecb-4de6-b7fc-66f1a03ae599(bp-sr0-port/backplane) -> c6354b9d-9749-4b03-b989-672c3dd497b8(tier0-01-tier1-01-t1_lrp/uplink)
172.31.1.2 > 192.168.1.2 10.31.1.2 > 192.168.1.2
02:50:56:56:44:52 > 02:50:56:56:53:00 02:50:56:56:44:55 > 02:50:56:56:44:52
'Networking' 카테고리의 다른 글
BFD Configuration on Tier-0 and Router (0) | 2023.09.25 |
---|---|
BFD Overview (0) | 2023.09.25 |
[NSX] Setup - 8. Tier-1 Gateway 생성 및 네트워크 통신 테스트 (0) | 2023.09.16 |
[NSX] Setup - 7. Tier-0 Gateway 생성 및 BGP 연결 (0) | 2023.09.16 |
[NSX] Setup - 6. Edge Uplink Profile 생성과 Edge 설치 (0) | 2023.09.16 |