NSX의 North-South Connectivity를 위해서 사용되는 BGP Protocol의 상태 점검 및 관련 로그들을 살펴보겠습니다.
참고로 NSX 환경에서, 현재는 BGP Protocol 뿐만 아니라 OSPF도 공식 Dynamic Protocol로 지원되고 있습니다.
실질적으로는 NSX의 Edge Node에 SR(Service Router)에 생성되는 Uplink Interface가 BGP Peer를 맺기 위한 Interface로 활용되며, BGP 상태도 해당 SR에서 확인이 가능합니다.
지난 번 구성했던 LAB 환경에서 BGP 정보들을 확인해보겠습니다.
구성 환경에 대한 정보는 아래 가이드를 참고하시기 바랍니다.
https://haewon83.tistory.com/176?category=1040732
https://haewon83.tistory.com/177?category=1040732
1. BGP 상태 확인
우선 Edge Node에 admin session으로 연결한 후, Logical Router 정보를 조회합니다.
| edge-node-01> get logical-routers Sat Jan 20 2024 UTC 06:33:50.663 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 4/5000 4f22c0b3-4a2f-4840-a8d2-cf8c797c087b 1 1 DR-Tier0-01 DISTRIBUTED_ROUTER_TIER0 5 2/50000 1eefa746-7662-4e21-8431-39dfc1f57394 2 2 SR-Tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 d533b216-a47a-4200-9eb3-007e68c3a024 4 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 5bc895a7-8cc3-4332-9b1a-abe3bd64fa22 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 96486497-be55-4cc3-8ae1-bbc7fe391d4b 6 11 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 |
상단 Router와 연결되는 Tier-0 SR의 VRF 번호를 확인하고 해당 SR로 이동한 다음 관련 Interface 목록을 확인합니다.
| edge-node-01> vrf 2 edge-node-01(tier0_sr[2])> get interface Sat Jan 20 2024 UTC 06:35:52.507 Logical Router UUID VRF LR-ID Name Type 1eefa746-7662-4e21-8431-39dfc1f57394 2 2 SR-Tier0-01 SERVICE_ROUTER_TIER0 Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable) Interface : c9755577-209f-4850-8510-65c63d8d388c Ifuid : 286 Name : edge01-uplink02 Fwd-mode : IPV4_ONLY Internal name : uplink-286 Mode : lif Port-type : uplink IP/Mask : 192.168.13.11/24 MAC : 00:50:56:a6:ab:70 VLAN : 1613 Access-VLAN : untagged LS port : 4e65dc04-96bd-4925-97a3-b3d84deaae02 Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : feb991cf-7a43-572b-a78f-094909d51633 Ifuid : 272 Mode : cpu Port-type : cpu Enable-mcast : false Interface : 99d172cf-bb05-5906-83c1-573897c09f68 Ifuid : 273 Mode : blackhole Port-type : blackhole Interface : bb0a4ec2-2b78-4fce-a37c-40b7c78792af Ifuid : 284 Mode : loopback Port-type : loopback IP/Mask : 127.0.0.1/8;::1/128(NA) Interface : 9afaf8e0-8f00-4204-bce5-f87a7d9f2946 Ifuid : 279 Name : edge01-uplink01 Fwd-mode : IPV4_ONLY Internal name : uplink-279 Mode : lif Port-type : uplink IP/Mask : 192.168.12.11/24 MAC : 00:50:56:a6:8c:14 VLAN : 1612 Access-VLAN : untagged LS port : 3f0de8d1-d706-4eab-bef7-af8dd836c10c Urpf-mode : STRICT_MODE DAD-mode : LOOSE RA-mode : SLAAC_DNS_TRHOUGH_RA(M=0, O=0) Admin : up Op_state : up Enable-mcast : False MTU : 1500 arp_proxy : Interface : ccfdcefa-4282-44a0-b89e-2a5654f9491e Ifuid : 281 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Internal name : downlink-281 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/24;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 65536 Access-VLAN : untagged LS port : aecc5e23-82b2-4f6c-a994-2bd47330a188 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy : |
Interface 들 중 Port-type이 uplink로 정의된 Interface UUID는 다음과 같습니다.
| Interface : c9755577-209f-4850-8510-65c63d8d388c Interface : 9afaf8e0-8f00-4204-bce5-f87a7d9f2946 |
Interface를 통해서 BGP Packet 확인은 다음에 보기로 하고, 우선 BGP 상태 정보를 확인해보겠습니다.
아래 정보를 보면, 상단 Router에 설정된 BGP Peer IP Address는 192.168.12.1과 192.168.13.1이고, Remote AS 번호가 65002로 확인됩니다.
또한, BGP 현재 상태는 Established로 확인되며, BFD는 설정되어 있지 않습니다.
| edge-node-01(tier0_sr[2])> get bgp neighbor summary BFD States: NC - Not configured, DC - Disconnected AD - Admin down, DW - Down, IN - Init, UP - Up BGP summary information for VRF default for address-family: ipv4Unicast Router ID: 192.168.12.11 Local AS: 65000 Neighbor AS State Up/DownTime BFD InMsgs OutMsgs InPfx OutPfx 192.168.12.1 65002 Estab 1d21h55m NC 2894 2894 0 3 192.168.13.1 65002 Estab 1d21h55m NC 2896 2894 0 3 Sat Jan 20 2024 UTC 06:39:41.425 |
위 내용은 NSX Manager UI에서도 다음과 같이 확인할 수 있습니다.
Networking > Tier-0 Gateways > Three Dot Ellipsis > Generate BGP Summary
BGP Neighbor IP Address를 이용하여 보다 상세한 조회도 가능합니다.
Hold Timer와 Keepalive Interval도 확인되는 것을 알 수 있습니다.
| edge-node-01(tier0_sr[2])> get bgp neighbor 192.168.12.1 BGP neighbor is 192.168.12.1, remote AS 65002, local AS 65000, external link Hostname: vyos01 BGP version 4, remote router ID 192.168.13.1, local router ID 192.168.12.11 BGP state = Established, up for 1d21h59m Last read 00:00:22, Last write 00:00:30 Hold time is 180, keepalive interval is 60 seconds Configured hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: 4 Byte AS: advertised and received AddPath: IPv4 Unicast: RX advertised IPv4 Unicast and received Route refresh: advertised and received(old & new) Address Family IPv4 Unicast: advertised and received Hostname Capability: advertised (name: edge-node-01,domain name: n/a) received (name: vyos01,domain name: n/a) Graceful Restart Capability: advertised and received Remote Restart timer is 120 seconds Address families by peer: none Graceful restart information: End-of-RIB send: IPv4 Unicast End-of-RIB received: IPv4 Unicast Local GR Mode: Helper* Remote GR Mode: Helper R bit: False Timers: Configured Restart Time(sec): 180 Received Restart Time(sec): 120 IPv4 Unicast: F bit: False End-of-RIB sent: Yes End-of-RIB sent after update: Yes End-of-RIB received: Yes Timers: Configured Stale Path Time(sec): 600 Message statistics: Inq depth is 0 Outq depth is 0 Sent Rcvd Opens: 2 2 Notifications: 0 0 Updates: 8 9 Keepalives: 2888 2887 Route Refresh: 0 0 Capability: 0 0 Total: 2898 2898 Minimum time between advertisement runs is 0 seconds Update source is 192.168.12.11 For address family: IPv4 Unicast Update group 3, subgroup 1 Packet Queue length 0 Community attribute sent to this neighbor(all) 0 accepted prefixes Connections established 2; dropped 1 Looped AS count 4 Last reset 1d21h59m, due to Peer closed the session Local host: 192.168.12.11, Local port: 44039 Foreign host: 192.168.12.1, Foreign port: 179 Nexthop: 192.168.12.11 Nexthop global: :: Nexthop local: :: BGP connection: shared network BGP Connect Retry Timer in Seconds: 10 Estimated round trip time: 2 ms Read thread: on Write thread: on FD used: 26 |
동시에 해당 정보를 상단 Router에서도 확인을 해야 합니다.
※ 본 LAB 환경에서는 VYOS를 사용
| vyos@vyos01:~$ show ip bgp summary IPv4 Unicast Summary: BGP router identifier 192.168.13.1, local AS number 65002 vrf-id 0 BGP table version 12 RIB entries 5, using 920 bytes of memory Peers 4, using 82 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.12.11 4 65000 2803 2797 0 0 0 1d22h27m 3 192.168.12.12 4 65000 2797 2796 0 0 0 1d22h28m 3 192.168.13.11 4 65000 2797 2795 0 0 0 1d22h28m 3 192.168.13.12 4 65000 2797 2797 0 0 0 1d22h28m 3 Total number of neighbors 4 vyos@vyos01:~$ show ip bgp BGP table version is 12, local router ID is 192.168.13.1, vrf id 0 Default local pref 100, local AS 65002 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *= 172.31.1.0/24 192.168.12.11 0 0 65000 ? *> 192.168.13.11 0 0 65000 ? * 192.168.12.12 0 0 65000 65000 65000 65000 ? * 192.168.13.12 0 0 65000 65000 65000 65000 ? *= 192.168.12.0/24 192.168.12.11 0 0 65000 ? *> 192.168.13.11 0 0 65000 ? * 192.168.12.12 0 0 65000 65000 65000 65000 ? * 192.168.13.12 0 0 65000 65000 65000 65000 ? *= 192.168.13.0/24 192.168.12.11 0 0 65000 ? *> 192.168.13.11 0 0 65000 ? * 192.168.12.12 0 0 65000 65000 65000 65000 ? * 192.168.13.12 0 0 65000 65000 65000 65000 ? Displayed 3 routes and 12 total paths |
2. BGP Packet 확인
BGP Connection이 맺어지고 나서는 BGP Peer 간 정상 여부를 확인하기 위해 Keepalive Interval 마다 Message를 주고 받습니다. 이를 SR에 위치한 Uplink Interface를 통해서 확인할 수 있습니다.
위에서 확인한 SR에 생성된 Uplink Interface의 UUID를 이용하여 Packet 수집을 진행해보겠습니다.
BGP Protocol은 179번 포트를 사용하므로 수집 시 Filter Option에 179 포트 번호를 이용하였습니다.
| edge-node-01> get logical-routers Sat Jan 20 2024 UTC 06:33:50.663 Logical Router UUID VRF LR-ID Name Type Ports Neighbors 736a80e3-23f6-5a2d-81d6-bbefb2786666 0 0 TUNNEL 4 4/5000 4f22c0b3-4a2f-4840-a8d2-cf8c797c087b 1 1 DR-Tier0-01 DISTRIBUTED_ROUTER_TIER0 5 2/50000 1eefa746-7662-4e21-8431-39dfc1f57394 2 2 SR-Tier0-01 SERVICE_ROUTER_TIER0 6 2/50000 d533b216-a47a-4200-9eb3-007e68c3a024 4 9 SR-tier1-01 SERVICE_ROUTER_TIER1 5 2/50000 5bc895a7-8cc3-4332-9b1a-abe3bd64fa22 5 8 DR-tier1-01 DISTRIBUTED_ROUTER_TIER1 4 0/50000 96486497-be55-4cc3-8ae1-bbc7fe391d4b 6 11 SR-one-arm SERVICE_ROUTER_TIER1 5 1/50000 edge-node-01> start capture interface c9755577-209f-4850-8510-65c63d8d388c expression port 179 06:50:53.220683 00:50:56:a6:ab:70 > 00:50:56:05:ae:d2, ethertype 802.1Q (0x8100), length 89: vlan 0, p 0, ethertype IPv4, 192.168.13.11.179 > 192.168.13.1.34268: Flags [P.], seq 4267751271:4267751290, ack 3251158562, win 255, options [nop,nop,TS val 1340868823 ecr 2499133717], length 19: BGP <base64>AFBWBa7SAFBWpqtwgQAAAAgARcAARwTwQAABBtikwKgNC8CoDQEAs4Xc/mC3Z8HIviKAGAD/ayEAAAEBCApP7AjXlPXBFf////////////////////8AEwQ=</base64> 06:50:53.220948 00:50:56:05:ae:d2 > 00:50:56:a6:ab:70, ethertype IPv4 (0x0800), length 66: 192.168.13.1.34268 > 192.168.13.11.179: Flags [.], ack 19, win 501, options [nop,nop,TS val 2499186439 ecr 1340868823], length 0 <base64>AFBWpqtwAFBWBa7SCABFwAA0UaNAAAEGjATAqA0BwKgNC4XcALPByL4i/mC3eoAQAfWgUwAAAQEICpT2jwdP7AjX</base64> |
상단 Router에서도 동일하게 port 179에 대한 Packet을 확인할 수 있습니다.
| vyos@vyos01:~$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:ae:d5 brd ff:ff:ff:ff:ff:ff inet 192.168.10.1/24 brd 192.168.10.255 scope global eth8 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe05:aed5/64 scope link valid_lft forever preferred_lft forever 3: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:ae:e9 brd ff:ff:ff:ff:ff:ff inet6 fe80::250:56ff:fe05:aee9/64 scope link valid_lft forever preferred_lft forever 4: eth11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:ae:d2 brd ff:ff:ff:ff:ff:ff inet6 fe80::250:56ff:fe05:aed2/64 scope link valid_lft forever preferred_lft forever 5: eth9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:ae:db brd ff:ff:ff:ff:ff:ff inet 192.168.11.1/24 brd 192.168.11.255 scope global eth9 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe05:aedb/64 scope link valid_lft forever preferred_lft forever 6: eth6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:23:ed brd ff:ff:ff:ff:ff:ff inet6 fe80::250:56ff:fe05:23ed/64 scope link valid_lft forever preferred_lft forever 7: eth7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:05:ae:d9 brd ff:ff:ff:ff:ff:ff inet 192.168.1.1/24 brd 192.168.1.255 scope global eth7 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe05:aed9/64 scope link valid_lft forever preferred_lft forever 9: eth10.1612@eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:50:56:05:ae:e9 brd ff:ff:ff:ff:ff:ff inet 192.168.12.1/24 brd 192.168.12.255 scope global eth10.1612 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe05:aee9/64 scope link valid_lft forever preferred_lft forever 10: eth11.1613@eth11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:50:56:05:ae:d2 brd ff:ff:ff:ff:ff:ff inet 192.168.13.1/24 brd 192.168.13.255 scope global eth11.1613 valid_lft forever preferred_lft forever inet6 fe80::250:56ff:fe05:aed2/64 scope link valid_lft forever preferred_lft forever vyos@vyos01:~$ tcpdump -i eth11.1613 tcp port 179 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth11.1613, link-type EN10MB (Ethernet), capture size 262144 bytes 06:54:52.818746 IP 192.168.13.12.35739 > vyos01.bgp: Flags [P.], seq 2260446665:2260446684, ack 1606435707, win 251, options [nop,nop,TS val 3858813099 ecr 720326578], length 19: BGP 06:54:52.818796 IP vyos01.bgp > 192.168.13.12.35739: Flags [.], ack 19, win 508, options [nop,nop,TS val 720380364 ecr 3858813099], length 0 |
Uplink Interface에서 수집한 Packet을 확장자 pcap으로 저장하여 Wireshark에서 확인해 볼 수 있습니다.
Peer 각각이 Keepalive Message를 보내는 것이 확인됩니다.
각 Peer가 Keepalive Message를 보내는 Interval도 60초로 확인됩니다.(Configurable)
| 1 0.000000 192.168.13.11 192.168.13.1 BGP 89 KEEPALIVE Message 2 0.000543 192.168.13.1 192.168.13.11 TCP 66 34268 → 179 [ACK] Seq=1 Ack=20 Win=501 Len=0 TSval=2499846441 TSecr=1341528827 3 7.279585 192.168.13.1 192.168.13.11 BGP 85 KEEPALIVE Message 4 7.280732 192.168.13.11 192.168.13.1 TCP 70 179 → 34268 [ACK] Seq=20 Ack=20 Win=255 Len=0 TSval=1341536107 TSecr=2499853719 |
3. BGP 관련 로그
NSX의 Edge에서 사용되는 BGP Protocol은 FRR을 이용하는 것으로 추정됩니다.(아직 관련 문서를 정확히 파악하지는 못했습니다.)
따라서, BGP Protocol과 관련된 Message들은 기본적으로 /var/log/frr 폴더 하위에 있는 frr.log에 기록됩니다.
192.168.13.1(상단 Router)과 192.168.13.11 간의 BGP Peering을 맺는 과정에 대한 로그가 확인됩니다.
※ 항상 어떤 Component가 되었든지, 이런 식으로 정상 상황의 로그들을 정리해야 문제가 생기는 시점의 로그를 비교/대조 할 수 있습니다.
| 2024/01/18 06:36:32.362583 BGP: 192.168.13.1 [FSM] Timer (start timer expire). 2024/01/18 06:36:32.362624 BGP: 192.168.13.1 [FSM] BGP_Start (Idle->Connect), fd -1 2024/01/18 06:36:32.362699 BGP: 192.168.13.1 [FSM] Waiting for NHT 2024/01/18 06:36:32.362710 BGP: bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 1 2024/01/18 06:36:32.362715 BGP: 192.168.13.1 (0x160e7e9c8f00 -1) went from Idle to Connect 2024/01/18 06:36:32.362729 BGP: 192.168.13.1 [FSM] TCP_connection_open_failed (Connect->Active), fd -1 2024/01/18 06:36:32.363139 ZEBRA: zebra_rnh_store_in_routing_table: 0:192.168.13.1/32 added for tracking on 0.0.0.0/0 2024/01/18 06:36:32.363169 ZEBRA: zebra_rnh_resolve_nexthop_entry: 0:192.168.13.1/32 Possible Match to 192.168.13.0/24 2024/01/18 06:36:32.363175 ZEBRA: Route Entry bgp !selected 2024/01/18 06:36:32.363183 ZEBRA: zebra_rnh_remove_from_routing_table: 0:192.168.13.1/32 removed from tracking on 0.0.0.0/0 2024/01/18 06:36:32.363192 ZEBRA: zebra_rnh_store_in_routing_table: 0:192.168.13.1/32 added for tracking on 192.168.13.0/24 2024/01/18 06:36:32.363447 BGP: bgp_fsm_change_status : vrf default(0), Status: Active established_peers 1 2024/01/18 06:36:32.363455 BGP: 192.168.13.1 (0x160e7e9c8f00 -1) went from Connect to Active 2024/01/18 06:36:32.363489 BGP: 192.168.13.1 [FSM] ConnectRetry_timer_expired (Active->Connect), fd -1 2024/01/18 06:36:32.363601 BGP: 192.168.13.1 [Event] Connect start to 192.168.13.1 fd 30 2024/01/18 06:36:32.363683 BGP: 192.168.13.1 [FSM] Non blocking connect waiting result, fd 30 2024/01/18 06:36:32.363693 BGP: bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 1 2024/01/18 06:36:32.363696 BGP: 192.168.13.1 (0x160e7e9c8f00 30) went from Active to Connect 2024/01/18 06:36:32.365994 BGP: 192.168.13.1 [FSM] TCP_connection_open (Connect->OpenSent), fd 30 2024/01/18 06:36:32.366035 BGP: 192.168.13.1 open active, local address 192.168.13.11 2024/01/18 06:36:32.366045 BGP: 192.168.13.1 Sending hostname cap with hn = edge-node-01, dn = (null) 2024/01/18 06:36:32.366051 BGP: 192.168.13.1 sending OPEN, version 4, my as 65000, holdtime 180, id 192.168.12.11 2024/01/18 06:36:32.366077 BGP: bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 1 2024/01/18 06:36:32.366083 BGP: 192.168.13.1 (0x160e7e9c8f00 30) went from Connect to OpenSent 2024/01/18 06:36:32.367221 BGP: 192.168.13.1 rcv OPEN, version 4, remote-as (in open) 65002, holdtime 180, id 192.168.13.1 2024/01/18 06:36:32.367231 BGP: 192.168.13.1 rcv OPEN w/ OPTION parameter len: 50 2024/01/18 06:36:32.367235 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 2024/01/18 06:36:32.367239 BGP: 192.168.13.1 OPEN has MultiProtocol Extensions capability (1), length 4 2024/01/18 06:36:32.367243 BGP: 192.168.13.1 OPEN has MP_EXT CAP for afi/safi: IPv4/unicast 2024/01/18 06:36:32.367248 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 2024/01/18 06:36:32.367251 BGP: 192.168.13.1 OPEN has Route Refresh (Old) capability (128), length 0 2024/01/18 06:36:32.367254 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 2024/01/18 06:36:32.367258 BGP: 192.168.13.1 OPEN has Route Refresh capability (2), length 0 2024/01/18 06:36:32.367261 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 2024/01/18 06:36:32.367264 BGP: 192.168.13.1 OPEN has 4-octet AS number capability (65), length 4 2024/01/18 06:36:32.367267 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 2024/01/18 06:36:32.367270 BGP: 192.168.13.1 OPEN has AddPath capability (69), length 4 2024/01/18 06:36:32.367274 BGP: 192.168.13.1 OPEN has AddPath CAP for afi/safi: IPv4/unicast, receive 2024/01/18 06:36:32.367277 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 10 2024/01/18 06:36:32.367281 BGP: 192.168.13.1 OPEN has FQDN capability (73), length 8 2024/01/18 06:36:32.367285 BGP: 192.168.13.1 received hostname vyos01, domainname (null) 2024/01/18 06:36:32.367289 BGP: 192.168.13.1 rcvd OPEN w/ optional parameter type 2 (Capability) len 4 2024/01/18 06:36:32.367292 BGP: 192.168.13.1 OPEN has Graceful Restart capability (64), length 2 2024/01/18 06:36:32.367295 BGP: 192.168.13.1 OPEN has Graceful Restart capability 2024/01/18 06:36:32.367298 BGP: 192.168.13.1 Peer has restarted. Restart Time : 120 2024/01/18 06:36:32.367311 BGP: 192.168.13.1 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 30 2024/01/18 06:36:32.367351 BGP: bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 1 2024/01/18 06:36:32.367356 BGP: 192.168.13.1 (0x160e7e9c8f00 30) went from OpenSent to OpenConfirm 2024/01/18 06:36:32.367359 BGP: 192.168.13.1: peer keepalive being added, acquiring lock 2024/01/18 06:36:32.367388 BGP: 192.168.13.1: peer keepalive added 2024/01/18 06:36:32.367622 BGP: 192.168.13.1 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 30 2024/01/18 06:36:32.367630 BGP: bgp_fsm_change_status : vrf default(0), Status: Established established_peers 2 2024/01/18 06:36:32.367633 BGP: 192.168.13.1 (0x160e7e9c8f00 30) went from OpenConfirm to Established 2024/01/18 06:36:32.367636 BGP: Peer 192.168.13.1 fd 30 send BGP_UP message to BGP adapter 2024/01/18 06:36:32.367700 BGP: %ADJCHANGE: neighbor 192.168.13.1(vyos01) in vrf default Up 2024/01/18 06:36:32.367722 BGP: peer 192.168.13.1 BGP_HELPER_MODE 2024/01/18 06:36:32.367768 BGP: 192.168.13.1 [FSM] Timer (routeadv timer expire) 2024/01/18 06:36:32.367826 BGP: BGP Adapter: Send BGP_UP for 192.168.13.1 |
이와 달리 문제가 생기는 경우의 frr.log 예를 한 번 보겠습니다.
BGP의 Hold Timer가 만료되면서 BGP Connection에 문제가 생기는 경우에 기록된 로그 입니다.
| 2024/01/08 07:18:51.235534 BGP: x.x.x.17 [FSM] Timer (holdtime timer expire) 2024/01/08 07:18:51.235594 BGP: x.x.x.17 [FSM] Hold_Timer_expired (Established->Clearing), fd 30 2024/01/08 07:18:51.235597 BGP: x.x.x.17 [FSM] Hold timer expire 2024/01/08 07:18:51.235626 BGP: %NOTIFICATION: sent to neighbor x.x.x.17 4/0 (Hold Timer Expired) 0 bytes 2024/01/08 07:18:51.235655 BGP: %ADJCHANGE: neighbor x.x.x.17(Unknown) in vrf default Down BGP Notification send 2024/01/08 07:18:51.235661 BGP: x.x.x.17: peer keepalive being removed, acquiring lock 2024/01/08 07:18:51.235664 BGP: x.x.x.17: peer keepalive removed 2024/01/08 07:18:51.235742 BGP: x.x.x.17(0x150ff92c81d0): close file descriptor 2024/01/08 07:18:51.235868 BGP: x.x.x.17 (0x150ff92c81d0 -1) went from Established to Clearing 2024/01/08 07:18:51.235879 BGP: Peer x.x.x.17 fd -1 send BGP_DOWN message to BGP adapter 2024/01/08 07:18:51.235903 BGP: BGP Adapter: Send BGP_DOWN for peer x.x.x.17 (vrf: default) |
위와 같은 경우에는 실제 Peer가 Keepalive Message를 전달해주지 않은 것인지 아니면, 내 Keepalive Message가 Peer가 전달되지 않은 것인지 문제 시점에 구간 내 Packet 수집을 통해 확인이 필요합니다.
NSX에서 사용되고 있는 Dynamic Routing Protocol 중 하나인 BGP의 상태 정보를 어떻게 조회하고, Packet과 Log를 확인하는 방법에 대해서 알아보았습니다.
다음에 환경 구성이 가능하다면, OSPF Protocol을 사용하는 환경에서는 어떤 내용들을 확인할 수 있을지 알아보도록 하겠습니다.
'Networking' 카테고리의 다른 글
| [NSX] Gateway Firewall (0) | 2024.01.28 |
|---|---|
| [NSX] Distributed Firewall (1) | 2024.01.26 |
| [NSX] Logical Routing (1) | 2024.01.15 |
| [NSX] Basic Check #3 (1) | 2023.12.11 |
| [NSX] Basic Check #2 (0) | 2023.12.11 |
