4. Interface 별 MTU 값 확인
| edge-node-01> get interface Tue Dec 05 2023 UTC 14:57:19.533 Interface: bond0 IPv4: Address: unknown Broadcast address: None MAC address: de:bb:15:af:b5:12 MTU: 1500 ^^^^^^^^^ KNI: False Bond mode: ACTIVE_BACKUP Bond primary slave: None Bond currently active slave: None Bond slaves: Link status: down Admin status: up RX packets: 0 RX bytes: 0 RX errors: 0 RX dropped: 0 TX packets: 0 TX bytes: 0 TX errors: 0 TX dropped: 0 TX collisions: 0 Interface: eth0 IPv4: Address: 192.168.1.111/24 Default gateway: 192.168.1.1 Broadcast address: 192.168.1.255 MAC address: 00:50:56:a1:e5:27 MTU: 1500 ^^^^^^^^^ KNI: False Link status: up Admin status: up RX packets: 5511595 RX bytes: 400410236 RX errors: 0 RX dropped: 0 TX packets: 1162680 TX bytes: 198835409 TX errors: 0 TX dropped: 0 TX collisions: 0 Interface: fp-eth0 ID: 0 Link status: up MAC address: 00:50:56:a1:4f:a5 MTU: 9000 ^^^^^^^^^ PCI: 0000:0b:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: active Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 46409703 RX bytes: 13609545659 RX errors: 0 RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 4202539 TX bytes: 552509951 TX errors: 0 TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth1 ID: 1 Link status: up MAC address: 00:50:56:a1:86:fa MTU: 9000 ^^^^^^^^^ PCI: 0000:13:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: active Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 47837735 RX bytes: 13945754969 RX errors: 0 RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 2774518 TX bytes: 216647987 TX errors: 0 TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth2 ID: 2 Link status: down MAC address: 00:50:56:a1:27:0f MTU: 1500 PCI: 0000:1b:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: inactive Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 0 RX bytes: 0 RX errors: 0 RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 0 TX bytes: 0 TX errors: 0 TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth3 ID: 3 Link status: down MAC address: 00:50:56:a1:a0:14 MTU: 1500 PCI: 0000:04:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: inactive Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 0 RX bytes: 0 RX errors: 0 RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 0 TX bytes: 0 TX errors: 0 TX pause xoff: unknown TX pause xon: unknown |
5. TEP Ping 테스트
Edge Node에서 TUNNEL VRF 확인
| edge-node-01> get logical-routers Tue Dec 05 2023 UTC 15:01:58.163 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 5/5000 ^^^^^^ 064b8812-743c-427a-b0b6-801570118070 1 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 5 0/50000 ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 92045476-c754-48e1-a86a-1ac4f9961112 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 0a35807f-c32a-4d6c-a58d-a0b73858fbf2 6 1026 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 |
Tier-0와 상단 Gateway간 TEP IP Address를 이용하여 Ping 테스트
| edge-node-01(vrf)> ping 192.168.11.2 source 192.168.11.4 dfbit enable size 2000 PING 192.168.11.2 (192.168.11.2) from 192.168.11.4: 2000 data bytes 2008 bytes from 192.168.11.2: icmp_seq=0 ttl=64 time=1.445 ms 2008 bytes from 192.168.11.2: icmp_seq=1 ttl=64 time=0.525 ms 2008 bytes from 192.168.11.2: icmp_seq=2 ttl=64 time=0.412 ms 2008 bytes from 192.168.11.2: icmp_seq=3 ttl=64 time=0.735 ms 2008 bytes from 192.168.11.2: icmp_seq=4 ttl=64 time=0.738 ms ^C --- 192.168.11.2 ping statistics --- 6 packets transmitted, 5 packets received, 16.7% packet loss round-trip min/avg/max/stddev = 0.412/0.771/1.445/0.359 ms |
6. Edge Node의 Tx, Rx Ring Size
| edge-node-01> get dataplane Tue Dec 05 2023 UTC 15:23:12.664 Accept_ra : False Bfd_ring_size : 512 Bitw_mode : False Corelist : 0,1,2,3,4,5 Crypto_broken_test_id: 0 Crypto_dequeue_burst_size: 32 Crypto_max_nb_qps : 8 Crypto_max_nb_sessions: 12000 Ctrl_prio_on : True Delay_first_probe_time_ms: 0 Dummy_rss_on : False Esp_rss_on : True Fc_max_mem_percore : 768 Fc_mega_cache_size : 262144 Fc_mega_hard_timeout_ms: 4985 Fc_mega_soft_timeout_ms: 4921 Fc_micro_cache_size: 262144 Firewall_flow_cache_on: 1 Firewall_max_purge_rate: 5 Firewall_max_sessions: 8388608 Firewall_purge_timeout_ms: 10 Firewall_timer_resolution_ms: 50 Flow_cache_mega : True Flow_cache_micro : True Flow_count_num_keys: 10000 Fwpurge_corelist : Geneve_cbit_set : False Hugepage_mem : 15360 Intr_mode_on : True Ip_reass_gc_period : 2 Ip_reass_interfrag_ms: 1000 Ip_reass_max_ms : 5000 Ip_reass_maxq_len : 45 Ip_reass_qhash_order: 8 Ip_reass_qlist_order: 9 Ipsec_sa_request_shift: 5 Is_bare_metal_edge : False Jumbo_mbuf_pool_size: 1024 Kni_busy_loop_enabled: 0 Kni_fifo_size : 1024 Kni_mbuf_burst_num : 256 Kni_queues_num : 4 L2vpn_pmtu_on : True Lacp_ring_size : 512 Learning_ring_size : 512 Link_speed : 0 Livetrace_ring_size: 512 Mainloop_sleep_threshold: 10 Mainloop_sleep_threshold_kni_max: 10000 Mainloop_sleep_timeout_max_ns: 1000000 Mainloop_sleep_timeout_min_ns: 1000 Mbuf_pool_size : 204538 Mem_mon_threshold : 85 N_mem_channel : 3 Nce_learn_delay_us : 0 Numa_on : True Oam_prioritization_enabled: True Pkt_queue_limit : 0 Pkt_queue_rx_burst_size: 128 Pkt_seg_pnic_limit : 24 Pmtu_expiry_in_min : 10 Pmtu_learning : True Prio_rx_queue_per_core: 32 Public_cloud_mode : False Qos_wakeup_interval_ms: 50 Queue_num_per_port_per_core: 1 Rss : c0=3:0:1:2/c1=3:0:1:2/c2=3:0:1:2/c3=3:0:1:2 Rss_balance_enabled: True Rss_post_ipsec : False Rx_queue_per_core : 32 Rx_ring_size : 2048 ^^^^^^^^^^^^^^^^^^^^^^^^^ Service_corelist : 4,5 ^^^^^^^^^^^^^^^^^^^^^^^^ Slowpath_ring_size : 512 ^^^^^^^^^^^^^^^^^^^^^^^^ Stale_timeout_default_ms: 0 Tx_ring_size : 4096 ^^^^^^^^^^^^^^^^^^^^^^^^^ Upt_mode : False Devices: Device_id : 0x07b0 Name : fp-eth0 Numa_node : 0 Pci : 0000:0b:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth1 Numa_node : 0 Pci : 0000:13:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth2 Numa_node : 0 Pci : 0000:1b:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth3 Numa_node : 0 Pci : 0000:04:00.00 Vendor : 0x15ad |
7. Edge CPU Utilization 확인
| root@edge-node-01:~# top top - 15:31:19 up 7 days, 20 min, 2 users, load average: 0.77, 0.44, 0.38 Tasks: 274 total, 1 running, 273 sleeping, 0 stopped, 0 zombie top - 15:31:39 up 7 days, 21 min, 2 users, load average: 0.70, 0.44, 0.38 Tasks: 273 total, 1 running, 272 sleeping, 0 stopped, 0 zombie %Cpu(s): 4.5 us, 3.0 sy, 0.0 ni, 92.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 32572368 total, 9567808 free, 16756376 used, 6248184 buff/cache KiB Swap: 0 total, 0 free, 0 used. 15556508 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ TGID COMMAND 1128579 nsx-sha 20 0 5796 576 516 S 0.0 0.0 0:00.00 1128579 sleep 1128398 root 20 0 16940 5828 5116 S 0.0 0.0 0:00.09 1128398 systemd-timedat 1127616 root 20 0 5836 704 644 S 0.0 0.0 0:00.00 1127616 mpstat 1127615 root 20 0 2620 592 528 S 0.0 0.0 0:00.00 1127615 sh 1127610 root 20 0 17708 11172 6312 S 0.0 0.0 0:00.03 1127610 python3 1127608 root 20 0 2616 1656 1568 S 0.0 0.0 0:00.00 1127608 sh 1127605 root 20 0 8476 2728 2296 S 0.0 0.0 0:00.00 1127605 cron 1126739 root 20 0 0 0 0 I 0.0 0.0 0:00.00 1126739 kworker/5:1-events 1126520 root 20 0 9708 3888 3108 R 0.7 0.0 0:00.89 1126520 top 1125834 root 20 0 0 0 0 I 0.0 0.0 0:00.00 1125834 kworker/7:0-cgroup_destroy 1125438 root 20 0 0 0 0 I 0.0 0.0 0:00.01 1125438 kworker/u16:2-flush-252:2 1125189 root 20 0 0 0 0 I 0.0 0.0 0:00.00 1125189 kworker/6:2-events 1124769 root 20 0 0 0 0 I 0.0 0.0 0:00.00 1124769 kworker/4:0-cgroup_destroy 1123669 root 20 0 0 0 0 I 0.0 0.0 0:00.00 1123669 kworker/5:0-cgroup_destroy 1117356 root 20 0 0 0 0 I 0.0 0.0 0:00.09 1117356 kworker/u16:4-flush-252:0 1113532 root 20 0 0 0 0 I 0.0 0.0 0:00.15 1113532 kworker/u16:1-events_unbound 1107975 root 20 0 0 0 0 I 0.0 0.0 0:00.20 1107975 kworker/u16:0-events_power_+ 1107940 root 20 0 0 0 0 I 0.0 0.0 0:00.11 1107940 kworker/7:2-mm_percpu_wq |
8. 외부 네트워크로 Ping 테스트
Tier-0의 Uplink Interface IP Address 확인
| edge-node-01> get logical-routers Tue Dec 05 2023 UTC 15:37:11.854 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 5/5000 064b8812-743c-427a-b0b6-801570118070 1 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 5 0/50000 ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 92045476-c754-48e1-a86a-1ac4f9961112 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 0a35807f-c32a-4d6c-a58d-a0b73858fbf2 6 1026 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 edge-node-01> vrf 3 edge-node-01(tier0_sr[3])> get interface Tue Dec 05 2023 UTC 15:37:29.972 Logical Router UUID VRF LR-ID Name Type ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 2e407500-2065-5637-a587-0d47d79b16ae Ifuid : 270 Mode : cpu Port-type : cpu Enable-mcast : false Interface : b293703d-622b-4cd7-9173-65efb2f07bd4 Ifuid : 292 Name : edge01-uplink01 ^^^^^^^^^^^^^^ Fwd-mode : IPV4_ONLY Internal name : uplink-292 Mode : lif Port-type : uplink ^^^^^^ IP/Mask : 192.168.12.11/24 ^^^^^^^^^^^^^^ MAC : 00:50:56:a1:4f:a5 VLAN : 1612 Access-VLAN : untagged LS port : 70273c6d-7580-4c0d-a09f-3f0a7cea42f1 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : 40557daa-b1a5-5d85-9935-d0a1d41bacaa Ifuid : 271 Mode : blackhole Port-type : blackhole Interface : 26b8d93c-bdd1-4155-b31f-5bf966009615 Ifuid : 279 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Internal name : backplane-279 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/24;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 72704 Access-VLAN : untagged LS port : fa66b101-63b9-45f0-9cff-dec67fb9c792 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 6b4feec9-7f27-4cce-be75-9c469cf56cbe Ifuid : 281 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : aeb063b7-c37c-43dd-a43f-44f97d92dabb Ifuid : 290 Name : edge01-uplink02 ^^^^^^^^^^^^^^ Fwd-mode : IPV4_ONLY Internal name : uplink-290 Mode : lif Port-type : uplink ^^^^^^ IP/Mask : 192.168.13.11/24 ^^^^^^^^^^^^^^ MAC : 00:50:56:a1:86:fa VLAN : 1613 Access-VLAN : untagged LS port : b32a861a-602f-41f9-97f8-7418c1e61649 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Logical Router UUID VRF LR-ID Name Type 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 7125be6d-f907-50dd-8f01-aedf79144354 Ifuid : 268 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 3906867b-bc94-5af8-bef2-d1dc67bd1707 Ifuid : 269 Mode : blackhole Port-type : blackhole Interface : ff510ac7-5993-4b6a-b244-4c4db22428d2 Ifuid : 278 Name : multicast-transit-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : mcast-transit IP/Mask : MAC : 02:50:56:56:44:52 VNI : 67584 Access-VLAN : untagged LS port : b3b805ed-3b44-44ea-b9ea-fe110baefdac Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : f6534080-6e5a-5f4a-9071-0758da2af81b Ifuid : 280 Name : tier0-01-tier1-01-t0_lrp Fwd-mode : IPV4_ONLY Internal name : downlink-280 Mode : lif Port-type : downlink IP/Mask : 100.64.0.0/31;fc14:9b49:677c:6c00::1/64(NA);fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 74753 Access-VLAN : untagged LS port : dbad468d-0a04-4924-a845-d0d65781688c Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : af319283-2f98-421b-bcd1-ebc5f8aa440b Ifuid : 284 Name : bp-dr-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.1/24;fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 72704 Access-VLAN : untagged LS port : 1663db89-7f1f-41b0-92ec-7f41be359137 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : |
Uplink IP Address를 이용하여 Ping 테스트
| edge-node-01(tier0_sr[3])> ping 192.168.1.1 source 192.168.12.11 PING 192.168.1.1 (192.168.1.1) from 192.168.12.11: 56 data bytes 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.942 ms 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.776 ms ^C --- 192.168.1.1 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/stddev = 0.776/0.859/0.942/0.083 ms edge-node-01(tier0_sr[3])> ping 192.168.1.1 source 192.168.13.11 PING 192.168.1.1 (192.168.1.1) from 192.168.13.11: 56 data bytes 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.561 ms 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.480 ms ^C --- 192.168.1.1 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/stddev = 0.480/0.520/0.561/0.041 ms edge-node-01(tier0_sr[3])> ping 192.168.1.1 source 192.168.12.11 dfbit enable size 1472 PING 192.168.1.1 (192.168.1.1) from 192.168.12.11: 1472 data bytes 1480 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=1.475 ms 1480 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.677 ms 1480 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=1.021 ms 1480 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=1.043 ms ^C --- 192.168.1.1 ping statistics --- 5 packets transmitted, 4 packets received, 20.0% packet loss round-trip min/avg/max/stddev = 0.677/1.054/1.475/0.283 ms edge-node-01(tier0_sr[3])> ping 192.168.1.1 source 192.168.13.11 dfbit enable size 1472 PING 192.168.1.1 (192.168.1.1) from 192.168.13.11: 1472 data bytes 1480 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.840 ms 1480 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.648 ms 1480 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.883 ms 1480 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=1.145 ms ^C --- 192.168.1.1 ping statistics --- 5 packets transmitted, 4 packets received, 20.0% packet loss round-trip min/avg/max/stddev = 0.648/0.879/1.145/0.177 ms |
'Networking' 카테고리의 다른 글
| [NSX] Logical Routing (1) | 2024.01.15 |
|---|---|
| [NSX] Basic Check #3 (1) | 2023.12.11 |
| [NSX] Basic Check #1 (1) | 2023.12.11 |
| Load Balancer Packet 수집 (0) | 2023.11.05 |
| Load Balancer 정보 조회 (0) | 2023.11.05 |
