NSX 구성 환경에서 외부 네트워크와 Overlay 네트워크 간 Ping을 통해 ICMP Packet을 주고 받을 때, 어떤 경로를 통해 주고 받는지를 살펴보겠습니다.
외부 네트워크부터 Edge의 Uplink 상단 네트워크까지는 제외하고 Edge로 들어온 이후부터 VM까지의 실제 Packet을 확인하는 과정입니다.
우선 Edge에서 Logical Router 및 Interface 정보를 확인합니다.
| edge-node-02> get logical-routers Sat Jan 13 2024 UTC 09:37:26.608 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 6/5000 22072e12-d79b-4e02-8094-fbc2a05525a5 1 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 1/50000 bb15fb90-2f87-4d51-bc7b-895b7f6f1785 2 3 SR-Tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 fd22a1b7-cd26-4c38-8db1-080235e6b6ce 4 11 SR-one-arm SERVICE_ROUTER_TIER1 5 2/50000 1e706525-1e60-4540-b407-87e71b574a27 5 1 DR-Tier0-01 DISTRIBUTED_ROUTER_TIER0 5 2/50000 2587c886-fe47-4954-9a28-99bf83866fa0 6 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 Logical Router UUID VRF LR-ID Name Type 1e706525-1e60-4540-b407-87e71b574a27 5 1 DR-Tier0-01 DISTRIBUTED_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 18faae06-ecab-5b9b-9ca0-2424a0d1a740 Ifuid : 271 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 72e8a3fa-7725-5c86-8c4b-deb00fc56171 Ifuid : 272 Mode : blackhole Port-type : blackhole Interface : 6c9e27c6-9a96-4623-996a-c9015fa63dc3 Ifuid : 291 Name : bp-dr-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.1/24;fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 68608 Access-VLAN : untagged LS port : cc4478e5-ce6e-42ea-89a6-475d68c76872 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 7cb4b3cc-47d0-49d7-9643-42043201f875 Ifuid : 287 Name : multicast-transit-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : mcast-transit IP/Mask : MAC : 02:50:56:56:44:52 VNI : 65536 Access-VLAN : untagged LS port : 59a52510-9591-4f30-a0c7-a54535aa3682 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 8866ff61-9a15-5e60-953b-d458fd12a80a Ifuid : 283 Name : Tier0-01-tier1-01-t0_lrp Fwd-mode : IPV4_ONLY Internal name : downlink-283 Mode : lif Port-type : downlink IP/Mask : 100.64.0.0/31;fe80::50:56ff:fe56:4452/64(NA);fc8f:8a0d:2baf:a800::1/64(NA) MAC : 02:50:56:56:44:52 VNI : 65538 Access-VLAN : untagged LS port : 8bbe9b6a-f67e-48c3-9645-26ca901378d8 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Logical Router UUID VRF LR-ID Name Type bb15fb90-2f87-4d51-bc7b-895b7f6f1785 2 3 SR-Tier0-01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : f8076649-a5b0-52f1-9d92-88dc9aaefed6 Ifuid : 265 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 0e5fc03d-425d-59fa-bae5-0a930b6b30ac Ifuid : 266 Mode : blackhole Port-type : blackhole Interface : 5fc5fd73-1112-49df-bf11-4f062fccac1f Ifuid : 285 Name : edge02-uplink01 Fwd-mode : IPV4_ONLY Internal name : uplink-285 Mode : lif Port-type : uplink IP/Mask : 192.168.12.12/24 MAC : 00:50:56:a1:2f:53 VLAN : 1612 Access-VLAN : untagged LS port : 5448a9ee-332e-47de-8127-e5e29f9382ec Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : 90a1839a-57dc-44dd-819f-c35641eabef6 Ifuid : 297 Name : bp-sr1-port Fwd-mode : IPV4_ONLY Internal name : backplane-297 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/24;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 68608 Access-VLAN : untagged LS port : fd2d8a44-bbd4-4684-9b11-a5b55352b13a Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 370891d2-5ada-4dd6-be70-f19a38149a49 Ifuid : 300 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : 808adc0a-1586-460e-b88d-2309e39d8613 Ifuid : 295 Name : edge02-uplink02 Fwd-mode : IPV4_ONLY Internal name : uplink-295 Mode : lif Port-type : uplink IP/Mask : 192.168.13.12/24 MAC : 00:50:56:a1:2f:4f VLAN : 1613 Access-VLAN : untagged LS port : 5314bcd0-1999-40a2-bdca-2a54ac03577a Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Logical Router UUID VRF LR-ID Name Type 2587c886-fe47-4954-9a28-99bf83866fa0 6 9 SR-tier1-01 SERVICE_ROUTER_TIER1 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : a1b0a67f-fcf3-5b51-920f-eeed8e44351f Ifuid : 273 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 51bbd9a7-c26c-5ea1-a8eb-54c9b30bfc57 Ifuid : 274 Mode : blackhole Port-type : blackhole Interface : 454a0212-cd6b-4ec3-86bb-d0419141ce71 Ifuid : 301 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : 66710558-937b-41b9-93e1-0dbb9e9ea1de Ifuid : 281 Name : Tier0-01-tier1-01-t1_lrp Fwd-mode : IPV4_ONLY Mode : lif Port-type : uplink IP/Mask : 100.64.0.1/31;fc8f:8a0d:2baf:a800::2/64(NA);fe80::50:56ff:fe56:4455/64(NA) MAC : 02:50:56:56:44:55 VNI : 65538 Access-VLAN : untagged LS port : a509de29-fceb-4e15-bddf-95b5243160d6 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : 70467247-bd34-4330-904b-ebf9f6334a50 Ifuid : 299 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/28;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 65539 Access-VLAN : untagged LS port : 4b3840eb-f1ff-4b59-8b0a-58650a67a0f7 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Logical Router UUID VRF LR-ID Name Type 22072e12-d79b-4e02-8094-fbc2a05525a5 1 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 8cb0700a-a10b-5082-88e9-106d78cb64eb Ifuid : 263 Mode : cpu Port-type : cpu Enable-mcast : false Interface : b5c3994d-5c90-53da-9067-d76ec24ca8ee Ifuid : 264 Mode : blackhole Port-type : blackhole Interface : f164e271-716d-479b-9197-ecee90ce4d59 Ifuid : 293 Name : infra-overlay-seg-3101-dlrp Fwd-mode : IPV4_ONLY Mode : lif Port-type : downlink IP/Mask : 172.31.1.1/24 MAC : 02:50:56:56:44:52 VNI : 69632 Access-VLAN : untagged LS port : 6c518cc1-83ff-47b5-9112-dd9c5df3b7a3 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 4a9a577c-e0dd-409a-819f-b988ca20fea9 Ifuid : 279 Name : bp-dr-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.1/28;fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 65539 Access-VLAN : untagged LS port : 779d7893-eb64-4db9-b11b-3497c1bd04cd Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : |
위에서 확인한 Logical Router별 Interface 정보를 보기 쉽게 표로 정리하였습니다.
Edge의 Router 및 Interface 정보
| Type | Interface | Port-type | IP/Mask | Internal name |
| SERVICE_ROUTER_TIER0 | 5fc5fd73-1112-49df-bf11-4f062fccac1f | uplink | 192.168.12.12/24 | uplink-285 |
| 808adc0a-1586-460e-b88d-2309e39d8613 | uplink | 192.168.13.12/24 | uplink-295 | |
| 90a1839a-57dc-44dd-819f-c35641eabef6 | backplane | 169.254.0.2/24 | backplane-297 | |
| DISTRIBUTED_ROUTER_TIER0 | 6c9e27c6-9a96-4623-996a-c9015fa63dc3 | backplane | 169.254.0.1/24 | bp-dr-port |
| 8866ff61-9a15-5e60-953b-d458fd12a80a | downlink | 100.64.0.0/31 | Tier0-01-tier1-01-t0_lrp | |
| SERVICE_ROUTER_TIER1 | 66710558-937b-41b9-93e1-0dbb9e9ea1de | uplink | 100.64.0.1/31 | Tier0-01-tier1-01-t1_lrp |
| 70467247-bd34-4330-904b-ebf9f6334a50 | backplane | 169.254.0.2/28 | bp-sr0-port | |
| DISTRIBUTED_ROUTER_TIER1 | f164e271-716d-479b-9197-ecee90ce4d59 | downlink | 172.31.1.1/24 | infra-overlay-seg-3101-dlrp |
| 4a9a577c-e0dd-409a-819f-b988ca20fea9 | backplane | 169.254.0.1/28 | bp-dr-port |
Client와 Server IP 정보
Client : 192.168.1.2
Server : 172.31.1.14
이제 확인한 Interface 정보를 토대로 Interface 별로 ICMP Request/Reply Packet을 확인해보겠습니다.
1. ICMP Request Packet Flow
1-1. Tier-0 SR의 uplink interface에서 ICMP Packet 확인
| edge-node-02> start capture interface 5fc5fd73-1112-49df-bf11-4f062fccac1f direction input expression icmp 12:54:23.311316 00:50:56:a1:a8:5a > 00:50:56:a1:2f:53, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.14: ICMP echo request, id 1, seq 10736, length 40 <base64>AFBWoS9TAFBWoahaCABFAAA8x4wAAH8BBV3AqAECrB8BDggAI2sAASnwYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
1-2. Tier-0 DR의 downlink interface에서 ICMP Packet 확인
| edge-node-02> start capture interface 8866ff61-9a15-5e60-953b-d458fd12a80a direction output expression icmp 12:57:01.712841 02:50:56:56:44:52 > 02:50:56:56:44:55, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.14: ICMP echo request, id 1, seq 10892, length 40 <base64>AlBWVkRVAlBWVkRSCABFAAA8yCgAAH4BBcHAqAECrB8BDggAIs8AASqMYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
1-3. Tier-1 SR의 uplink interface에서 ICMP Packet 확인
| edge-node-02> start capture interface 66710558-937b-41b9-93e1-0dbb9e9ea1de direction input expression icmp 12:58:21.922250 02:50:56:56:44:52 > 02:50:56:56:44:55, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.14: ICMP echo request, id 1, seq 10971, length 40 <base64>AlBWVkRVAlBWVkRSCABFAAA8yHcAAH4BBXLAqAECrB8BDggAIoAAASrbYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
1-4. Tier-1 DR의 downlink interface에서 ICMP Packet 확인
| edge-node-02> start capture interface f164e271-716d-479b-9197-ecee90ce4d59 direction output expression icmp 12:59:45.175741 02:50:56:56:44:52 > 00:50:56:a1:ea:f0, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.14: ICMP echo request, id 1, seq 11053, length 40 <base64>AFBWoerwAlBWVkRSCABFAAA8yMkAAH0BBiDAqAECrB8BDggAIi4AASstYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
1-5. 여기서 Tier-1 DR의 downlink interface에 연결된 Logical Switch(Segment)를 통해 VM이 위치한 Transport Node로 이동하여 실제 ICMP Packet을 수신할 VM으로 Packet 전달
| [root@esxi701:~] net-stats -l PortNum Type SubType SwitchName MACAddress ClientName 2214592529 4 0 DvsPortset-0 00:50:56:a1:50:8b vmnic4 2214592531 4 0 DvsPortset-0 00:50:56:a1:fe:a8 vmnic3 67108887 3 0 DvsPortset-0 00:50:56:6d:16:1d vmk10 67108888 3 0 DvsPortset-0 00:50:56:6a:81:57 vmk11 67108890 3 0 DvsPortset-0 00:50:56:60:87:65 vmk50 67108891 0 0 DvsPortset-0 02:50:56:56:44:52 vdr-vdrPort 67108892 5 9 DvsPortset-0 00:50:56:a1:ea:f0 centos7.eth0 2248146959 4 0 DvsPortset-1 00:50:56:a1:49:19 vmnic2 2281701386 4 0 vSwitch0 00:50:56:a1:c3:7d vmnic0 134217749 3 0 vSwitch0 00:50:56:a1:c3:7d vmk0 134217753 3 0 vSwitch0 00:50:56:67:98:39 vmk2 134217757 5 7 vSwitch0 00:50:56:a1:e0:cd vrlcm01 2315255821 4 0 vSwitch1 00:50:56:a1:5e:c5 vmnic1 167772182 3 0 vSwitch1 00:50:56:6a:44:6e vmk1 [root@esxi701:~] pktcap-uw --switchport 67108892 --dir 1 -o - | tcpdump-uw -enr - icmp The switch port id is 0x0400001c. pktcap: The output file is -. pktcap: No server port specifed, select 40941 as the port. pktcap: Local CID 2. pktcap: Listen on port 40941. pktcap: Main thread: 862338743168. pktcap: Dump Thread: 862339278592. pktcap: Recv Thread: 862339806976. pktcap: Accept... pktcap: Vsock connection from port 1031 cid 2. reading from file -, link-type EN10MB (Ethernet) 13:01:59.765460 02:50:56:56:44:52 > 00:50:56:a1:ea:f0, ethertype IPv4 (0x0800), length 74: 192.168.1.2 > 172.31.1.14: ICMP echo request, id 1, seq 11167, length 40 |
2. ICMP Reply Packet Flow
ICMP Packet을 수신한 VM의 ICMP Stack에서 ICMP Reply Packet을 만들고 이는 Transport Node에 위치한 DR인 VDR로 전달
2-1. Transport Node의 VDR Instance에서 ICMP Packet 확인
| [root@esxi701:~] net-stats -l PortNum Type SubType SwitchName MACAddress ClientName 2214592529 4 0 DvsPortset-0 00:50:56:a1:50:8b vmnic4 2214592531 4 0 DvsPortset-0 00:50:56:a1:fe:a8 vmnic3 67108887 3 0 DvsPortset-0 00:50:56:6d:16:1d vmk10 67108888 3 0 DvsPortset-0 00:50:56:6a:81:57 vmk11 67108890 3 0 DvsPortset-0 00:50:56:60:87:65 vmk50 67108891 0 0 DvsPortset-0 02:50:56:56:44:52 vdr-vdrPort 67108892 5 9 DvsPortset-0 00:50:56:a1:ea:f0 centos7.eth0 2248146959 4 0 DvsPortset-1 00:50:56:a1:49:19 vmnic2 2281701386 4 0 vSwitch0 00:50:56:a1:c3:7d vmnic0 134217749 3 0 vSwitch0 00:50:56:a1:c3:7d vmk0 134217753 3 0 vSwitch0 00:50:56:67:98:39 vmk2 134217757 5 7 vSwitch0 00:50:56:a1:e0:cd vrlcm01 2315255821 4 0 vSwitch1 00:50:56:a1:5e:c5 vmnic1 167772182 3 0 vSwitch1 00:50:56:6a:44:6e vmk1 [root@esxi701:~] pktcap-uw --switchport 67108891 --dir 1 -o - | tcpdump-uw -enr - icmp The switch port id is 0x0400001b. pktcap: The output file is -. pktcap: No server port specifed, select 42016 as the port. pktcap: Local CID 2. pktcap: Listen on port 42016. pktcap: Main thread: 101009062784. pktcap: Dump Thread: 101009598208. pktcap: Recv Thread: 101010126592. pktcap: Accept... pktcap: Vsock connection from port 1035 cid 2. reading from file -, link-type EN10MB (Ethernet) 13:08:26.560706 00:50:56:a1:ea:f0 > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 11548, length 40 |
2-2. ICMP Reply Packet을 수신한 VDR은 Edge에 위치한 Tier-1 SR과 연결 통로인 Backplane을 통해 ICMP Packet 전달
| [root@esxi701:~] net-vdr -l -I DR Instance Information : --------------------------- DR UUID: 22072e12-d79b-4e02-8094-fbc2a05525a5 DR Id: 0x00000008 Number of Lifs: 2 Number of Routesv4: 3 Number of Routesv6: 5 Number of Hold Pkts: 0 State: Enabled Num unique nexthops: 1 Number of ARP Entries: 5 Max number of ARP Entries: 50000 Generation Number: 0 Edge Active: No Pmac: 00:00:00:00:00:00 Dynamic resource pool tag: 65535 Multicast Routing: Disabled Num MRouters: 0 [root@esxi701:~] net-vdr -l -L 22072e12-d79b-4e02-8094-fbc2a05525a5 DR 22072e12-d79b-4e02-8094-fbc2a05525a5 LIF Information : IPv6 DAD Status Legend: [A:SUCCESS], [F:FAILED], [T:TENTATIVE], [U:UNKNOWN] UUID: 4a9a577c-e0dd-409a-819f-b988ca20fea9 Mode: Routing-Backplane Id: Overlay:65539 Ipv4/Mask: 169.254.0.1/28 Ipv6/Mask: fe80::50:56ff:fe56:4452/128(U) Mac: 02:50:56:56:44:52 Connected Dvs: DSwitch-nsx-overlay VXLAN Control Plane: Enabled Replication Mode: 0.0.0.1 Multicast Routing: Enabled, Oper Down State: U Flags: 0x90308 DHCP Relay: Not enabled DAD Mode: LOOSE RA Mode: SLAAC_DNS_THROUGH_RA(M=0, O=0) URPF Mode: PORT CHECK UUID: f164e271-716d-479b-9197-ecee90ce4d59 Mode: Routing, Downlink Id: Overlay:69632 Ipv4/Mask: 172.31.1.1/24 Ipv6/Mask: Mac: 02:50:56:56:44:52 Connected Dvs: DSwitch-nsx-overlay VXLAN Control Plane: Enabled Replication Mode: 0.0.0.1 Multicast Routing: Enabled, Oper Down State: Enabled Flags: 0x80388 DHCP Relay: Not enabled DAD Mode: LOOSE RA Mode: UNKNOWN URPF Mode: STRICT [root@esxi701:~] net-vdr -l -L 22072e12-d79b-4e02-8094-fbc2a05525a5 --brief DR 22072e12-d79b-4e02-8094-fbc2a05525a5 LIF Information : State Legend: [A:Active], [d:Deleting], [X:Deleted], [I:Init],[SF-L:Soft Flush LIF] Modes Legend: [B:Bridging],[E: Empty],[R:Routing,R-L:Link,R-B:Backplane],[D:Distributed],[DL:Downlink] Id Mode State Ip/Mask Mac Lif UUID -- ----- ----- ------- ---- -------- Overlay:65539 R-B A 169.254.0.1/28 02:50:56:56:44:52 4a9a577c-e0dd-409a-819f-b988ca20fea9 fe80::50:56ff:fe56:4452/128 Overlay:69632 R,DL A 172.31.1.1/24 02:50:56:56:44:52 f164e271-716d-479b-9197-ecee90ce4d59 [root@esxi701:~] pktcap-uw --switchport 67108891 --dir 0 -o - | tcpdump-uw -enr - icmp The switch port id is 0x0400001b. pktcap: The output file is -. pktcap: No server port specifed, select 41992 as the port. pktcap: Local CID 2. pktcap: Listen on port 41992. pktcap: Main thread: 501068499840. pktcap: Dump Thread: 501069035264. pktcap: Recv Thread: 501069563648. pktcap: Accept... pktcap: Vsock connection from port 1034 cid 2. reading from file -, link-type EN10MB (Ethernet) 13:08:19.451362 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 11541, length 40 |
2-3. Tier-1 SR의 backplane interface에서 Tier-1 DR로부터 전달되는 ICMP Packet 확인
| edge-node-02> start capture interface 70467247-bd34-4330-904b-ebf9f6334a50 direction input expression icmp 13:13:05.188508 02:50:56:56:44:52 > 02:50:56:56:53:00, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 11841, length 40 <base64>AlBWVlMAAlBWVkRSCABFAAA8um4AAD8BUnusHwEOwKgBAgAAJxoAAS5BYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
2-4. Tier-1 SR의 uplink interface에서 Tier-0 DR로 전달되는 ICMP Packet 확인
| edge-node-02> start capture interface 66710558-937b-41b9-93e1-0dbb9e9ea1de direction output expression icmp 13:14:05.089180 02:50:56:56:44:55 > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 11900, length 40 <base64>AlBWVkRSAlBWVkRVCABFAAA8LVgAAD8B35GsHwEOwKgBAgAAJt8AAS58YWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
2-5. Tier-0 DR의 downlink interface에서 Tier-1 SR로부터 전달되는 ICMP Packet 확인
| edge-node-02> start capture interface 8866ff61-9a15-5e60-953b-d458fd12a80a direction input expression icmp 13:15:58.820235 02:50:56:56:44:55 > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 12012, length 40 <base64>AlBWVkRSAlBWVkRVCABFAAA8H+QAAD8B7QWsHwEOwKgBAgAAJm8AAS7sYWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
2-6. Tier-0 SR의 uplink interface에서 외부로 나가는 ICMP Packet 확인
| edge-node-02> start capture interface 808adc0a-1586-460e-b88d-2309e39d8613 direction output expression icmp 13:17:14.998200 00:50:56:a1:2f:4f > 00:50:56:a1:97:8a, ethertype IPv4 (0x0800), length 74: 172.31.1.14 > 192.168.1.2: ICMP echo reply, id 1, seq 12087, length 40 <base64>AFBWoZeKAFBWoS9PCABFAAA8s0MAAD4BWqasHwEOwKgBAgAAJiQAAS83YWJjZGVmZ2hpamtsbW5vcHFyc3R1dndhYmNkZWZnaGk=</base64> |
이 전체 과정을 도식화 하면 다음과 같습니다.
위 경우는 VM이 Tier-1 DR 하위에 생성된 Logical Switch에 연결되어 있고, Tier-1 SR 에 Service Instance가 생성된 경우입니다.
즉, 현재 구성과 다른 경우에는 위와 같은 Flow가 아닌 다른 Flow가 그려질 수도 있음을 감안해야 합니다.
하지만, 대부분 고객사 환경에 Multi-Tier 구조를 사용하고, Tier-1에 Service Instance도 사용하고 있기 때문에 위 구성과 크게 다르지 않을 것으로 생각됩니다.
Multi-Tier 환경에서 위와 같은 Flow를 이해한 후, 실제 Network Connectivity가 올바르지 않은 경우, 각 구간별 Interface에서 Packet 수집을 통해 어디서 이슈가 발생하는지를 파악해 볼 수 있습니다.
'Networking' 카테고리의 다른 글
| [NSX] Distributed Firewall (1) | 2024.01.26 |
|---|---|
| [NSX] BGP Basic Check (0) | 2024.01.20 |
| [NSX] Basic Check #3 (1) | 2023.12.11 |
| [NSX] Basic Check #2 (0) | 2023.12.11 |
| [NSX] Basic Check #1 (1) | 2023.12.11 |
