Networking
[NSX] NSX-T host configuration shows Validation Errors with error code "9564"
haewon83
2024. 6. 10. 12:21
Transport Node에 대해서 "Symtpom" Section에 기재한 오류 메시지가 발생하는 경우가 있어, 이를 해결한 방법을 알아보겠습니다.
[Symptom]
단일 클러스터 내 모든 Transport Node에 대해서 다음과 같은 오류 메시지 발생
9564: Unable to fetch host hardware and network compute information associated with <Host UUID> from inventory. Either the CM-Inventory Service is not running, or inventory information is missing.
[확인 내용]
1. KB94655에 기재된 대로 Transport Node에 대해 TNP(Transport Node Profile)을 Detach/Attach 하면 문제가 해결
2. 테스트를 통해 TNP Deatch/Attach는 Network Connectivity에 문제가 없음을 확인
2-1. Cluster에서 Detach TNP
2-2. TNP가 Detach 되었지만, Transport Node에는 아무 영향이 없음
| [root@comp-esxi-01:~] localcli network ip connection list | grep "1234\|1235" tcp 0 0 192.168.1.31:37966 192.168.1.41:1234 ESTABLISHED 265616 newreno nsx-proxy tcp 0 0 192.168.1.31:37964 192.168.1.41:1235 ESTABLISHED 265616 newreno nsx-proxy [root@comp-esxi-01:~] localcli software vib list | grep nsx nsx-adf 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-cfgagent 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-context-mux 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-cpp-libs 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-esx-datapath 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-exporter 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-host 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-ids 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-monitoring 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-mpa 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-nestdb 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-netopa 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-opsagent 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-platform-client 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-proto2-libs 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-proxy 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-python-gevent 1.1.0-18242523 VMware VMwareCertified 2024-05-18 nsx-python-greenlet 0.4.14-18242315 VMware VMwareCertified 2024-05-18 nsx-python-logging 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-python-protobuf 2.6.1-18242311 VMware VMwareCertified 2024-05-18 nsx-python-utils 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-sfhc 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-shared-libs 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsx-vdpi 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 nsxcli 3.2.4.0.0-7.0.23653567 VMware VMwareCertified 2024-05-18 |
2-3. TNP를 다시 Cluster에 적용
※ TNP를 Detach/Attach하는 동안 Dataplane에는 문제가 없음
3. 2번 단계를 진행하기 전에 NSX Manager 로그에 vCenter와의 연결에 문제가 있는 것으로 확인
| 2024-05-21T10:47:07.903Z WARN EamPollingThread VcUtils 4767 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] IOException occurred javax.net.ssl.SSLHandshakeException: 35deb362a43035bd826d27d5d5491c33521b67a1f05c18d79a5b610c3614bec0 at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:1.8.0_301] at sun.security.ssl.TransportContext.fatal(TransportContext.java:348) ~[?:1.8.0_301] at sun.security.ssl.TransportContext.fatal(TransportContext.java:291) ~[?:1.8.0_301] at sun.security.ssl.TransportContext.fatal(TransportContext.java:286) ~[?:1.8.0_301] at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654) ~[?:1.8.0_301] at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473) ~[?:1.8.0_301] at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369) ~[?:1.8.0_301] at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377) ~[?:1.8.0_301] at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) ~[?:1.8.0_301] at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422) ~[?:1.8.0_301] at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) ~[?:1.8.0_301] at sun.security.ssl.SSLTransport.decode(SSLTransport.java:156) ~[?:1.8.0_301] at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1409) ~[?:1.8.0_301] at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1315) ~[?:1.8.0_301] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:439) ~[?:1.8.0_301] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:410) ~[?:1.8.0_301] at sun.net.http://www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) ~[?:1.8.0_301] at sun.net.http://www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197) ~[?:1.8.0_301] at sun.net.http://www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570) ~[?:1.8.0_301] at sun.net.http://www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_301] at sun.net.http://www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268) ~[?:1.8.0_301] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcUtils.getVcVersionClass(VcUtils.java:182) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionImpl.loadKeyStore(VcConnectionImpl.java:248) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionImpl$VcConnectionBuilder.buildConnectionUsingLoginCredential(VcConnectionImpl.java:195) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionImpl$VcConnectionBuilder.build(VcConnectionImpl.java:229) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionManagerImpl.getVcConnection(VcConnectionManagerImpl.java:115) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionManagerImpl.getVcConnection(VcConnectionManagerImpl.java:88) ~[?:?] at cohttp://m.vmware.nsx.management.inventory.access.vc.VcConnectionManagerImpl.getVcEamConnection(VcConnectionManagerImpl.java:189) ~[?:?] at cohttp://m.vmware.nsx.management.lcm.vc.soap.service.VCSoapServiceImpl.getVcEamConnection(VCSoapServiceImpl.java:862) ~[?:?] at cohttp://m.vmware.nsx.management.lcm.vc.soap.service.VCSoapServiceImpl.getEamStatus(VCSoapServiceImpl.java:734) ~[?:?] at cohttp://m.vmware.nsx.management.service_fabric.sfdm.vc.service.VCOperationsServiceImpl.getEamStatus(VCOperationsServiceImpl.java:246) ~[?:?] at cohttp://m.vmware.nsx.management.service_fabric.sfm.deployment.hostpinned.service.HostPinnedExecuteOperationImpl.getEamStatus(HostPinnedExecuteOperationImpl.java:237) ~[?:?] at cohttp://m.vmware.nsx.management.lcm.vmdeployment.hostbased.service.AgencyMonitoringService$AgencyPollingService.updateEamStatusForComputeManagers(AgencyMonitoringService.java:260) ~[?:?] at cohttp://m.vmware.nsx.management.lcm.vmdeployment.hostbased.service.AgencyMonitoringService$AgencyPollingService.run(AgencyMonitoringService.java:179) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_301] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_301] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_301] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_301] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_301] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_301] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_301] Caused by: cohttp://m.vmware.nsx.management.security.ThumbprintMismatchException: 35deb362a43035bd826d27d5d5491c33521b67a1f05c18d79a5b610c3614bec0 at cohttp://m.vmware.nsx.management.security.NsxTrustManager.checkThumbprintTrusted(NsxTrustManager.java:381) ~[nsx-trustmanager-1.0.jar:?] at cohttp://m.vmware.nsx.management.security.NsxTrustManager.checkServerTrusted(NsxTrustManager.java:259) ~[nsx-trustmanager-1.0.jar:?] at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1256) ~[?:1.8.0_301] at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638) ~[?:1.8.0_301] ... 36 more |
4. vCenter의 갱신된 인증서가 NSX Manager에 반영되지 않았기 때문으로 파악되어, vCenter 인증서의 Thumbprint를 획득하여 NSX Manager에 업데이트
$ echo | openssl s_client -connect <vcenter>:443 2>/dev/null | openssl x509 -noout -fingerprint -sha256
System > Compute Managers > vCenter 선택 > EDIT
[References]
NSX-T host configuration shows Validation Errors with error code "9564"
https://knowledge.broadcom.com/external/article?legacyId=94655