NSX의 성능 이슈를 점검할 때, 기본으로 참고할 만한 내용들을 정리하였습니다.
향후에는 NSX Edge의 High CPU, High Memory등 x86 OS 관점에서 Network Packet Processing에 영향을 줄 수 있는 내용들을 다뤄보고자 합니다.
1. Dataplane Core Utilization
Edge Node에서 Core의 반은 Dataplane에 할당되고, 나머지 반은 Non-Dataplane에 할당
Dataplane에 할당된 Core의 CPU Utilization이 높은 경우, Packet Drop이 발생할 수 있음
모든 ECMP Edge Node와 Active Tier-1 Edge Node에서 아래 명령어 확인
| edge-node-01> get dataplane cpu stats Tue Dec 05 2023 UTC 08:34:59.852 CPU Usage Core : 0 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 0 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 1 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 20 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 2 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 20 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 3 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 20 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 4 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 0 pps Slowpath : 0 pps Tx : 0 pps Usage : n/a Core : 5 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 0 pps Slowpath : 0 pps Tx : 0 pps Usage : n/a edge-node-02> get dataplane cpu stats Tue Dec 05 2023 UTC 08:35:02.677 CPU Usage Core : 0 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 40 pps Slowpath : 10 pps Tx : 10 pps Usage : 0% Core : 1 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 80 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 2 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 20 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 3 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 30 pps Slowpath : 0 pps Tx : 0 pps Usage : 0% Core : 4 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 0 pps Slowpath : 0 pps Tx : 0 pps Usage : n/a Core : 5 Crypto : 0 pps Intercore : 0 pps Kni : 0 pps Rx : 0 pps Slowpath : 0 pps Tx : 0 pps Usage : n/a |
Dataplane에 할당된 Core의 Utilization이 90%를 넘는 경우, Dataplane Core에서 실행 중인 Service가 문제의 원인일 수 있기 때문에 해당 Service를 다른 Core에서 실행될 수 있도록 하는 방법 확인 필요
가용 CPU Core 확인
| root@edge-node-01:~# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 45 bits physical, 48 bits virtual CPU(s): 8 On-line CPU(s) list: 0-7 ### <-- !! ^^^^^^^^^^^^^^^^^^^ Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 8 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz Stepping: 7 CPU MHz: 2095.077 BogoMIPS: 4190.15 Hypervisor vendor: VMware Virtualization type: full L1d cache: 256 KiB L1i cache: 256 KiB L2 cache: 8 MiB L3 cache: 176 MiB NUMA node0 CPU(s): 0-7 Vulnerability Itlb multihit: KVM: Mitigation: VMX unsupported Vulnerability L1tf: Mitigation; PTE Inversion Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown Vulnerability Retbleed: Mitigation; Enhanced IBRS Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization (limited, manual) Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon ljmpq nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_time r aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat pku avx512_vnni md_clear flush _l1d arch_capabilities |
Dataplane에 의해서 사용 중인 Core 확인
| edge-node-01> get dataplane Tue Dec 05 2023 UTC 08:43:50.592 Accept_ra : False Bfd_ring_size : 512 Bitw_mode : False Corelist : 0,1,2,3,4,5 ### <-- !! ^^^^^^^^ Crypto_broken_test_id: 0 Crypto_dequeue_burst_size: 32 Crypto_max_nb_qps : 8 Crypto_max_nb_sessions: 12000 Ctrl_prio_on : True Delay_first_probe_time_ms: 0 Dummy_rss_on : False Esp_rss_on : True Fc_max_mem_percore : 768 Fc_mega_cache_size : 262144 Fc_mega_hard_timeout_ms: 4985 Fc_mega_soft_timeout_ms: 4921 Fc_micro_cache_size: 262144 Firewall_flow_cache_on: 1 Firewall_max_purge_rate: 5 Firewall_max_sessions: 8388608 Firewall_purge_timeout_ms: 10 Firewall_timer_resolution_ms: 50 Flow_cache_mega : True Flow_cache_micro : True Flow_count_num_keys: 10000 Fwpurge_corelist : Geneve_cbit_set : False Hugepage_mem : 15360 Intr_mode_on : True Ip_reass_gc_period : 2 Ip_reass_interfrag_ms: 1000 Ip_reass_max_ms : 5000 Ip_reass_maxq_len : 45 Ip_reass_qhash_order: 8 Ip_reass_qlist_order: 9 Ipsec_sa_request_shift: 5 Is_bare_metal_edge : False Jumbo_mbuf_pool_size: 1024 Kni_busy_loop_enabled: 0 Kni_fifo_size : 1024 Kni_mbuf_burst_num : 256 Kni_queues_num : 4 L2vpn_pmtu_on : True Lacp_ring_size : 512 Learning_ring_size : 512 Link_speed : 0 Livetrace_ring_size: 512 Mainloop_sleep_threshold: 10 Mainloop_sleep_threshold_kni_max: 10000 Mainloop_sleep_timeout_max_ns: 1000000 Mainloop_sleep_timeout_min_ns: 1000 Mbuf_pool_size : 204538 Mem_mon_threshold : 85 N_mem_channel : 3 Nce_learn_delay_us : 0 Numa_on : True Oam_prioritization_enabled: True Pkt_queue_limit : 0 Pkt_queue_rx_burst_size: 128 Pkt_seg_pnic_limit : 24 Pmtu_expiry_in_min : 10 Pmtu_learning : True Prio_rx_queue_per_core: 32 Public_cloud_mode : False Qos_wakeup_interval_ms: 50 Queue_num_per_port_per_core: 1 Rss : c0=3:0:1:2/c1=3:0:1:2/c2=3:0:1:2/c3=3:0:1:2 Rss_balance_enabled: True Rss_post_ipsec : False Rx_queue_per_core : 32 Rx_ring_size : 2048 Service_corelist : 4,5 Slowpath_ring_size : 512 Stale_timeout_default_ms: 0 Tx_ring_size : 4096 Upt_mode : False Devices: Device_id : 0x07b0 Name : fp-eth0 Numa_node : 0 Pci : 0000:0b:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth1 Numa_node : 0 Pci : 0000:13:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth2 Numa_node : 0 Pci : 0000:1b:00.00 Vendor : 0x15ad Device_id : 0x07b0 Name : fp-eth3 Numa_node : 0 Pci : 0000:04:00.00 Vendor : 0x15ad |
kni_single Kernel Thread의 PID 확인
| root@edge-node-01:~# ps -aux|grep -i [k]ni_single root 4977 0.7 0.0 0 0 ? S Nov28 74:26 [kni_single] |
kni_single Kernel Thread의 Core Affinity 확인
| root@edge-node-01:~# taskset -pc 4977 pid 4977's current affinity list: 7 |
2. Tier-0 Router Interface Stats
Tier-0 SR Instance 확인
| edge-node-01> get logical-routers Tue Dec 05 2023 UTC 14:26:01.424 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 5/5000 064b8812-743c-427a-b0b6-801570118070 1 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 5 0/50000 ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 ^^^ ^^^^^^^^^^^ 92045476-c754-48e1-a86a-1ac4f9961112 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 0a35807f-c32a-4d6c-a58d-a0b73858fbf2 6 1026 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 |
Tier-0 Uplink Interface 확인
| edge-node-01> vrf 3 edge-node-01(tier0_sr[3])> get interface Tue Dec 05 2023 UTC 14:27:49.786 Logical Router UUID VRF LR-ID Name Type ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 ^^^ ^^^^^^^^^^^ Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 2e407500-2065-5637-a587-0d47d79b16ae Ifuid : 270 Mode : cpu Port-type : cpu Enable-mcast : false Interface : b293703d-622b-4cd7-9173-65efb2f07bd4 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Ifuid : 292 Name : edge01-uplink01 ^^^^^^^^^^^^^^^ Fwd-mode : IPV4_ONLY Internal name : uplink-292 ^^^^^^^^^^ Mode : lif Port-type : uplink IP/Mask : 192.168.12.11/24 MAC : 00:50:56:a1:4f:a5 VLAN : 1612 Access-VLAN : untagged LS port : 70273c6d-7580-4c0d-a09f-3f0a7cea42f1 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : 40557daa-b1a5-5d85-9935-d0a1d41bacaa Ifuid : 271 Mode : blackhole Port-type : blackhole Interface : 26b8d93c-bdd1-4155-b31f-5bf966009615 Ifuid : 279 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Internal name : backplane-279 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/24;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 72704 Access-VLAN : untagged LS port : fa66b101-63b9-45f0-9cff-dec67fb9c792 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : 6b4feec9-7f27-4cce-be75-9c469cf56cbe Ifuid : 281 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : aeb063b7-c37c-43dd-a43f-44f97d92dabb ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Ifuid : 290 Name : edge01-uplink02 ^^^^^^^^^^^^^^^ Fwd-mode : IPV4_ONLY Internal name : uplink-290 ^^^^^^^^^^ Mode : lif Port-type : uplink IP/Mask : 192.168.13.11/24 MAC : 00:50:56:a1:86:fa VLAN : 1613 Access-VLAN : untagged LS port : b32a861a-602f-41f9-97f8-7418c1e61649 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Logical Router UUID VRF LR-ID Name Type 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : 7125be6d-f907-50dd-8f01-aedf79144354 Ifuid : 268 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 3906867b-bc94-5af8-bef2-d1dc67bd1707 Ifuid : 269 Mode : blackhole Port-type : blackhole Interface : ff510ac7-5993-4b6a-b244-4c4db22428d2 Ifuid : 278 Name : multicast-transit-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : mcast-transit IP/Mask : MAC : 02:50:56:56:44:52 VNI : 67584 Access-VLAN : untagged LS port : b3b805ed-3b44-44ea-b9ea-fe110baefdac Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : Interface : f6534080-6e5a-5f4a-9071-0758da2af81b Ifuid : 280 Name : tier0-01-tier1-01-t0_lrp Fwd-mode : IPV4_ONLY Internal name : downlink-280 Mode : lif Port-type : downlink IP/Mask : 100.64.0.0/31;fc14:9b49:677c:6c00::1/64(NA);fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 74753 Access-VLAN : untagged LS port : dbad468d-0a04-4924-a845-d0d65781688c Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : SLAAC_DNS_THROUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : af319283-2f98-421b-bcd1-ebc5f8aa440b Ifuid : 284 Name : bp-dr-port Fwd-mode : IPV4_ONLY Mode : lif Port-type : backplane IP/Mask : 169.254.0.1/24;fe80::50:56ff:fe56:4452/64(NA) MAC : 02:50:56:56:44:52 VNI : 72704 Access-VLAN : untagged LS port : 1663db89-7f1f-41b0-92ec-7f41be359137 Urpf-mode : PORT_CHECK DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : |
Tier-0의 Uplink Interface 별 통계치 확인
| edge-node-01> get logical-router interface b293703d-622b-4cd7-9173-65efb2f07bd4 stats Tue Dec 05 2023 UTC 14:34:50.862 interface : b293703d-622b-4cd7-9173-65efb2f07bd4 ifuid : 292 VRF : ec318044-8bd6-4269-aa47-544ee747f765 name : edge01-uplink01 IP/Mask : 192.168.12.11/24 MAC : 00:50:56:a1:4f:a5 VLAN : 1612 LS port : 70273c6d-7580-4c0d-a09f-3f0a7cea42f1 urpf-mode : STRICT_MODE admin : up op_state : up MTU : 1500 statistics RX-Packets : 1388308 RX-Bytes : 91435804 RX-Drops : 75475 ^^^^^^^^ Blocked : 0 DST-Unsupported: 0 Firewall : 0 Malformed : 0 No-Receiver : 0 No-Route : 0 RPF-Check : 0 Protocol-Unsupported: 75474 ### <-- !! IPv6 : 1 Port-Unsupported: 0 TTL-Exceeded: 0 Kni : 0 Rate-Limit : 0 IPsec : 0 IPsec-NoSA : 0 IPsec-NoVTI : 0 TX-Packets : 1388447 TX-Bytes : 90738605 TX-Drops : 35132 ^^^^^^^^ Blocked : 0 Firewall : 0 Frag-Needed : 0 No-neighbor : 35132 ### <-- !! No-Memory : 0 No-Linked-Port: 0 Rate-Limit : 0 IPsec : 0 IPsec-NoSA : 0 IPsec-NoVTI : 0 IPsec-Policy-Error: 0 IPsec-Policy-Block: 0 IP Ressemble Fragments-OK: 0 Fragemnts-Error: 0 Fragments-Timeout: 0 IP Fragment Fragments-OK: 0 Fragments-Error: 0 edge-node-01> get logical-router interface aeb063b7-c37c-43dd-a43f-44f97d92dabb stats Tue Dec 05 2023 UTC 14:35:29.276 interface : aeb063b7-c37c-43dd-a43f-44f97d92dabb ifuid : 290 VRF : ec318044-8bd6-4269-aa47-544ee747f765 name : edge01-uplink02 IP/Mask : 192.168.13.11/24 MAC : 00:50:56:a1:86:fa VLAN : 1613 LS port : b32a861a-602f-41f9-97f8-7418c1e61649 urpf-mode : STRICT_MODE admin : up op_state : up MTU : 1500 statistics RX-Packets : 1388595 RX-Bytes : 91454596 RX-Drops : 75492 ^^^^^^^^ Blocked : 0 DST-Unsupported: 0 Firewall : 0 Malformed : 0 No-Receiver : 0 No-Route : 0 RPF-Check : 0 Protocol-Unsupported: 75491 ### <-- !! IPv6 : 1 Port-Unsupported: 0 TTL-Exceeded: 0 Kni : 0 Rate-Limit : 0 IPsec : 0 IPsec-NoSA : 0 IPsec-NoVTI : 0 TX-Packets : 1388315 TX-Bytes : 90728689 TX-Drops : 35161 ^^^^^^^^ Blocked : 0 Firewall : 0 Frag-Needed : 0 No-neighbor : 35161 ### <-- !! No-Memory : 0 No-Linked-Port: 0 Rate-Limit : 0 IPsec : 0 IPsec-NoSA : 0 IPsec-NoVTI : 0 IPsec-Policy-Error: 0 IPsec-Policy-Block: 0 IP Ressemble Fragments-OK: 0 Fragemnts-Error: 0 Fragments-Timeout: 0 IP Fragment Fragments-OK: 0 Fragments-Error: 0 |
fp-eth Interface 별 통계치 확인
| edge-node-01> get interface Tue Dec 05 2023 UTC 14:38:19.400 Interface: bond0 IPv4: Address: unknown Broadcast address: None MAC address: de:bb:15:af:b5:12 MTU: 1500 KNI: False Bond mode: ACTIVE_BACKUP Bond primary slave: None Bond currently active slave: None Bond slaves: Link status: down Admin status: up RX packets: 0 RX bytes: 0 RX errors: 0 RX dropped: 0 ^^^^^^^^^^ TX packets: 0 TX bytes: 0 TX errors: 0 TX dropped: 0 ^^^^^^^^^^ TX collisions: 0 Interface: eth0 IPv4: Address: 192.168.1.111/24 Default gateway: 192.168.1.1 Broadcast address: 192.168.1.255 MAC address: 00:50:56:a1:e5:27 MTU: 1500 KNI: False Link status: up Admin status: up RX packets: 5502702 RX bytes: 399711766 RX errors: 0 RX dropped: 0 ^^^^^^^^^^ TX packets: 1159927 TX bytes: 198338987 TX errors: 0 TX dropped: 0 ^^^^^^^^^^ TX collisions: 0 Interface: fp-eth0 ID: 0 Link status: up MAC address: 00:50:56:a1:4f:a5 MTU: 9000 PCI: 0000:0b:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: active Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 46323625 RX bytes: 13583015068 RX errors: 0 ^^^^^^^^^ RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 4194379 TX bytes: 551406960 TX errors: 0 ^^^^^^^^^ TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth1 ID: 1 Link status: up MAC address: 00:50:56:a1:86:fa MTU: 9000 PCI: 0000:13:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: active Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 47748801 RX bytes: 13918535946 RX errors: 0 ^^^^^^^^^ RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 2769212 TX bytes: 216233404 TX errors: 0 ^^^^^^^^^ TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth2 ID: 2 Link status: down MAC address: 00:50:56:a1:27:0f MTU: 1500 PCI: 0000:1b:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: inactive Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 0 RX bytes: 0 RX errors: 0 ^^^^^^^^^ RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 0 TX bytes: 0 TX errors: 0 ^^^^^^^^^ TX pause xoff: unknown TX pause xon: unknown Interface: fp-eth3 ID: 3 Link status: down MAC address: 00:50:56:a1:a0:14 MTU: 1500 PCI: 0000:04:00:00 Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO TX_MULTI_SEGS RX_VLAN_STRIP RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO Polling Status: inactive Driver: net_vmxnet3 Rx queue: 5 Tx queue: 4 Socket: 0 RX packets: 0 RX bytes: 0 RX errors: 0 ^^^^^^^^^ RX badcrc: unknown RX badlen: unknown RX misses: 0 RX nombufs: 0 RX pause xoff: unknown RX pause xon: unknown TX packets: 0 TX bytes: 0 TX errors: 0 ^^^^^^^^^ TX pause xoff: unknown TX pause xon: unknown |
3. Tier-0 BGP 상태 확인
| edge-node-01> get logical-routers Tue Dec 05 2023 UTC 14:42:45.272 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 5/5000 064b8812-743c-427a-b0b6-801570118070 1 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 73bdcf75-0251-4927-9c80-68a6d7265911 2 1 DR-tier0-01 DISTRIBUTED_ROUTER_TIER0 5 0/50000 ec318044-8bd6-4269-aa47-544ee747f765 3 2 SR-tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 ^^^ ^^^^^^^^^^^ 92045476-c754-48e1-a86a-1ac4f9961112 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 0a35807f-c32a-4d6c-a58d-a0b73858fbf2 6 1026 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 edge-node-01> vrf 3 edge-node-01(tier0_sr[3])> get bgp neighbor summary Warning: your password will expire in 5 days { "ipv4Unicast":{ "routerId":"192.168.12.11", "as":65000, "vrfId":0, "vrfName":"default", "tableVersion":6, "ribCount":11, "ribMemory":2024, "peerCount":2, "peerMemory":45808, "peers":{ "192.168.12.1":{ "hostname":"vyos", "remoteAs":65002, "version":4, "msgRcvd":9550, "msgSent":9547, "tableVersion":0, "outq":0, "inq":0, "peerUptime":"6d14h58m", "peerUptimeMsec":572332000, "peerUptimeEstablishedEpoch":1701215081, "prefixReceivedCount":5, "pfxRcd":5, "pfxSnt":3, "state":"Established", ^^^^^^^^^^^^^^^^^^^^^ "peerBfdStatus":{ "status":"Up" }, "idType":"ipv4" }, "192.168.13.1":{ "hostname":"vyos", "remoteAs":65002, "version":4, "msgRcvd":9550, "msgSent":9547, "tableVersion":0, "outq":0, "inq":0, "peerUptime":"6d14h58m", "peerUptimeMsec":572332000, "peerUptimeEstablishedEpoch":1701215081, "prefixReceivedCount":5, "pfxRcd":5, "pfxSnt":3, "state":"Established", ^^^^^^^^^^^^^^^^^^^^^ "peerBfdStatus":{ "status":"Up" }, "idType":"ipv4" } }, "totalPeers":2, "dynamicPeers":0, "bestPath":{ "multiPathRelax":"true" } } } Tue Dec 05 2023 UTC 14:43:33.648 |
Routing Table 정보 확인
| edge-node-01(tier0_sr[3])> get route Warning: your password will expire in 5 days { "100.64.0.0\/31":[ { "prefix":"100.64.0.0\/31", "protocol":"connected", "selected":true, "destSelected":true, "distance":0, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d23h41m", "nexthops":[ { "flags":3, "fib":true, "directlyConnected":true, "interfaceIndex":23, "interfaceName":"downlink-280", "active":true } ] } ], "169.254.0.0\/24":[ { "prefix":"169.254.0.0\/24", "protocol":"connected", "selected":true, "destSelected":true, "distance":0, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d15h08m", "nexthops":[ { "flags":3, "fib":true, "directlyConnected":true, "interfaceIndex":22, "interfaceName":"backplane-279", "active":true } ] } ], "172.31.1.0\/24":[ { "prefix":"172.31.1.0\/24", "protocol":"static", "selected":true, "destSelected":true, "distance":3, "metric":0, "installed":true, "tag":4016, "table":254, "internalStatus":16, "internalFlags":72, "uptime":"6d23h41m", "nexthops":[ { "flags":3, "fib":true, "ip":"100.64.0.1", "afi":"ipv4", "interfaceIndex":23, "interfaceName":"downlink-280", "active":true } ] } ], "192.168.1.0\/24":[ { "prefix":"192.168.1.0\/24", ^^^^^^^^^^^^^^^^^^^^^^^^^^ "protocol":"bgp", ^^^^^^^^^^^^^^^^ "selected":true, "destSelected":true, "distance":20, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d15h08m", "nexthops":[ ^^^^^^^^^^ { "flags":3, "fib":true, "ip":"192.168.12.1", ^^^^^^^^^^^^^^^^^^^ "afi":"ipv4", "interfaceIndex":27, "interfaceName":"uplink-292", "active":true }, { "flags":3, "fib":true, "ip":"192.168.13.1", ^^^^^^^^^^^^^^^^^^^ "afi":"ipv4", "interfaceIndex":25, "interfaceName":"uplink-290", "active":true } ] } ], "192.168.10.0\/24":[ { "prefix":"192.168.10.0\/24", "protocol":"bgp", "selected":true, "destSelected":true, "distance":20, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d15h08m", "nexthops":[ { "flags":3, "fib":true, "ip":"192.168.12.1", "afi":"ipv4", "interfaceIndex":27, "interfaceName":"uplink-292", "active":true }, { "flags":3, "fib":true, "ip":"192.168.13.1", "afi":"ipv4", "interfaceIndex":25, "interfaceName":"uplink-290", "active":true } ] } ], "192.168.11.0\/24":[ { "prefix":"192.168.11.0\/24", "protocol":"bgp", "selected":true, "destSelected":true, "distance":20, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d15h08m", "nexthops":[ { "flags":3, "fib":true, "ip":"192.168.12.1", "afi":"ipv4", "interfaceIndex":27, "interfaceName":"uplink-292", "active":true }, { "flags":3, "fib":true, "ip":"192.168.13.1", "afi":"ipv4", "interfaceIndex":25, "interfaceName":"uplink-290", "active":true } ] } ], "192.168.12.0\/24":[ { "prefix":"192.168.12.0\/24", "protocol":"connected", "selected":true, "destSelected":true, "distance":0, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d23h41m", "nexthops":[ { "flags":3, "fib":true, "directlyConnected":true, "interfaceIndex":27, "interfaceName":"uplink-292", "active":true } ] } ], "192.168.13.0\/24":[ { "prefix":"192.168.13.0\/24", "protocol":"connected", "selected":true, "destSelected":true, "distance":0, "metric":0, "installed":true, "table":254, "internalStatus":16, "internalFlags":8, "uptime":"6d23h41m", "nexthops":[ { "flags":3, "fib":true, "directlyConnected":true, "interfaceIndex":25, "interfaceName":"uplink-290", "active":true } ] } ] } Tue Dec 05 2023 UTC 14:53:13.778 |
'Networking' 카테고리의 다른 글
| [NSX] Basic Check #3 (1) | 2023.12.11 |
|---|---|
| [NSX] Basic Check #2 (0) | 2023.12.11 |
| Load Balancer Packet 수집 (0) | 2023.11.05 |
| Load Balancer 정보 조회 (0) | 2023.11.05 |
| One-arm Load Balancer 생성 (0) | 2023.10.30 |