Compute
[NUMA] numactl and numastat
haewon83
2023. 6. 11. 21:50
CentOS VM 테스트 환경 구성
Physical Machine
- 2 CPU package
- 32 cores per a CPU package
- 256GB Physical Memory
Virtual Machine
- vCPU 32
- Memory 192GB
- coresPerSocket = 1
NUMA 정보 확인
- 2 VPDs, 2 PPDs
- 2 NUMA Nodes
- VM GID는 307073
- VM의 vCPU는 NUMA Node0과 1에 16개씩 동일하게 분산
ESXi NUMA 정보 확인
| # vmdumper -l | cut -d \/ -f 2-5 | while read path; do egrep -oi "DICT.*(displayname.*|numa.*|cores.*|vcpu.*|memsize.*|affinity.*)= .*|numa:.*|numaHost:.*" "/$path/vmware.log"; echo -e; done DICT numvcpus = "32" DICT memSize = "196608" DICT sched.cpu.affinity = "all" DICT displayName = "centos79" DICT numa.autosize.cookie = "320012" DICT numa.autosize.vcpu.maxPerVirtualNode = "16" numaHost: NUMA config: consolidation= 1 preferHT= 0 partitionByMemory = 0 numaHost: 32 VCPUs 2 VPDs 2 PPDs numaHost: VCPU 0 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 1 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 2 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 3 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 4 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 5 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 6 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 7 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 8 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 9 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 10 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 11 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 12 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 13 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 14 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 15 VPD 0 PPD 0 NodeMask ffffffffffffffff numaHost: VCPU 16 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 17 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 18 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 19 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 20 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 21 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 22 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 23 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 24 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 25 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 26 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 27 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 28 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 29 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 30 VPD 1 PPD 1 NodeMask ffffffffffffffff numaHost: VCPU 31 VPD 1 PPD 1 NodeMask ffffffffffffffff [root@w2-tse-d12:~] sched-stats -t numa-pnode nodeID used idle entitled owed loadAvgPct nVcpu freeMem totalMem 0 3799 28194 580 0 0 35 82779612 133821428 1 447 31546 240 0 1 24 111270620 134217728 [root@w2-tse-d12:~] sched-stats -t numa-clients groupName groupID clientID homeNode affinity nWorlds vmmWorlds localMem remoteMem currLocal cummLocal vm.2101536 26406 0 1 3 8 8 16777212 0 100 99 vm.2111855 65772 0 0 3 2 2 8388608 0 100 99 vm.2145499 213912 0 0 3 2 2 4194304 0 100 99 vm.2148046 226863 0 0 3 8 8 8493056 45056 99 99 vm.2149367 235134 0 0 3 2 2 8388608 0 100 99 vm.2164552 294850 0 0 3 2 2 12582912 0 100 100 vm.2164620 294914 0 0 3 2 2 1814528 0 100 100 vm.2164844 296399 0 0 3 1 1 1048576 0 100 99 vm.2167067 307073 0 0 3 16 16 3176448 0 100 100 vm.2167067 307073 1 1 3 16 16 3774464 0 100 100 [root@w2-tse-d12:~] sched-stats -t numa-migration groupName groupID clientID balanceMig loadMig localityMig longTermMig monitorMig loadSwap localitySwap pageMigRate vm.2101536 26406 0 0 0 28 0 9 0 2 0 vm.2111855 65772 0 0 0 190 0 19 0 4 0 vm.2145499 213912 0 0 0 5 0 1 0 1 0 vm.2148046 226863 0 0 0 49 0 5 0 3 41 vm.2149367 235134 0 0 0 4 0 1 0 0 0 vm.2164552 294850 0 0 0 0 0 1 0 0 0 vm.2164620 294914 0 0 0 0 0 1 0 0 0 vm.2164844 296399 0 0 0 1 0 0 0 1 0 vm.2167067 307073 0 0 0 0 0 0 0 0 0 vm.2167067 307073 1 0 0 0 0 0 0 0 0 # esxtop 2:56:15pm up 3 days 23:45, 1477 worlds, 9 VMs, 59 vCPUs; CPU load average: 0.03, 0.05, 0.05 PCPU USED(%): 1.1 0.4 1.3 0.1 0.5 0.7 1.0 1.0 0.6 0.7 0.2 2.2 0.0 1.1 1.4 3.9 5.2 0.7 1.0 0.5 0.0 0.0 1.0 2.0 0.7 2.6 1.2 0.6 1.3 1.4 0.0 0.0 2.0 0.0 2.3 0.9 0.0 0.0 2.0 0.2 0.1 0.6 0.0 0.0 0.0 4.2 1.6 0.3 1.8 0.1 0.1 8.8 0.0 0.0 0.8 1.0 0.2 2.8 2.0 0.1 0.2 0.1 2.1 0.0 NUMA: 1.1 1.1 AVG: 1.1 PCPU UTIL(%): 1.0 0.4 1.2 0.1 0.5 0.7 0.9 1.0 0.6 0.6 0.2 1.9 0.0 1.0 1.3 3.2 4.3 0.7 0.9 0.4 0.0 0.0 0.9 1.9 0.6 2.2 1.0 0.5 1.3 1.3 0.0 0.0 1.7 0.0 1.9 0.8 0.0 0.0 1.8 0.2 0.1 0.5 0.0 0.0 0.0 3.5 1.4 0.2 1.5 0.1 0.1 6.9 0.0 0.0 0.7 0.8 0.2 2.4 1.6 0.1 0.2 0.1 1.8 0.0 NUMA: 1.0 0.9 AVG: 0.9 CORE UTIL(%): 1.4 1.2 1.0 1.8 1.2 1.5 1.0 4.4 4.8 1.3 0.1 2.7 2.8 1.5 2.4 0.0 1.7 2.7 0.1 1.9 0.5 0.0 3.5 1.6 1.6 6.9 0.0 1.5 2.5 1.7 0.0 1.6 NUMA: 1.8 1.7 AVG: 1.8 ID GID NAME NWLD %USED %RUN %SYS %WAIT %VMWAIT %RDY %IDLE %OVRLP %CSTP %MLMTD %SWPWT 26406 26406 ad01 18 20.54 16.22 0.07 1780.97 0.12 0.11 782.68 0.26 0.00 0.00 0.00 226863 226863 esxi703 21 8.78 7.08 0.12 2089.42 0.37 0.35 791.31 0.12 0.00 0.00 0.00 321671 321671 esxtop.2170365 1 8.77 6.78 0.00 93.07 - 0.00 0.00 0.00 0.00 0.00 0.00 65772 65772 vrops01 13 7.95 5.99 0.07 1292.54 0.11 0.13 193.87 0.03 0.00 0.00 0.00 294850 294850 vcsa01 26 6.56 5.16 0.06 2590.83 0.20 0.10 194.53 0.02 0.00 0.00 0.00 1 1 system 926 3.69 6343.21 0.00 86108.35 - 53.53 0.00 2.37 0.00 0.00 0.00 294914 294914 vcsa01-Passive 26 1.84 1.47 0.04 2594.53 0.27 0.09 198.23 0.02 0.00 0.00 0.00 235134 235134 vrops01-cp01 13 1.32 1.02 0.03 1296.91 0.51 0.11 198.43 0.02 0.00 0.00 0.00 2167067 307073 vmx 1 0.16 0.01 0.15 99.84 - 0.00 0.00 0.00 0.00 0.00 0.00 2167069 307073 NetWorld-VM-216 1 0.00 0.00 0.00 99.85 - 0.00 0.00 0.00 0.00 0.00 0.00 2167070 307073 NUMASchedRemapE 1 0.00 0.00 0.00 99.85 - 0.00 0.00 0.00 0.00 0.00 0.00 2167071 307073 vmast.2167068 1 0.08 0.06 0.00 99.79 - 0.00 0.00 0.00 0.00 0.00 0.00 2167103 307073 vmx-vthread-216 1 0.00 0.00 0.00 99.85 - 0.00 0.00 0.00 0.00 0.00 0.00 2167104 307073 vmx-filtPoll:ce 1 0.00 0.00 0.00 99.85 - 0.00 0.00 0.00 0.00 0.00 0.00 2167105 307073 vmx-mks:centos7 1 0.00 0.00 0.00 99.84 - 0.00 0.00 0.00 0.00 0.00 0.00 2167106 307073 vmx-svga:centos 1 0.00 0.00 0.00 99.85 - 0.00 0.00 0.00 0.00 0.00 0.00 2167107 307073 vmx-vcpu-0:cent 1 0.05 0.05 0.00 99.80 0.58 0.01 99.22 0.00 0.00 0.00 0.00 2167109 307073 vmx-vcpu-1:cent 1 0.07 0.06 0.00 99.78 1.26 0.01 98.52 0.00 0.00 0.00 0.00 2167110 307073 vmx-vcpu-2:cent 1 0.02 0.01 0.00 99.83 12.86 0.00 86.97 0.00 0.00 0.00 0.00 2167111 307073 vmx-vcpu-3:cent 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167112 307073 vmx-vcpu-4:cent 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167113 307073 vmx-vcpu-5:cent 1 0.01 0.01 0.00 99.84 11.68 0.01 88.15 0.00 0.00 0.00 0.00 2167114 307073 vmx-vcpu-6:cent 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167115 307073 vmx-vcpu-7:cent 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167116 307073 vmx-vcpu-8:cent 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167117 307073 vmx-vcpu-9:cent 1 0.01 0.01 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167118 307073 vmx-vcpu-10:cen 1 0.40 0.34 0.00 99.45 0.03 0.06 99.42 0.01 0.00 0.00 0.00 2167119 307073 vmx-vcpu-11:cen 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167120 307073 vmx-vcpu-12:cen 1 0.07 0.06 0.00 99.78 0.58 0.01 99.20 0.00 0.00 0.00 0.00 2167121 307073 vmx-vcpu-13:cen 1 0.01 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167122 307073 vmx-vcpu-14:cen 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167123 307073 vmx-vcpu-15:cen 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167124 307073 vmx-vcpu-16:cen 1 0.02 0.02 0.00 99.83 8.21 0.00 91.62 0.00 0.00 0.00 0.00 2167125 307073 vmx-vcpu-17:cen 1 0.02 0.02 0.00 99.83 5.60 0.01 94.22 0.00 0.00 0.00 0.00 2167126 307073 vmx-vcpu-18:cen 1 0.02 0.02 0.00 99.83 3.19 0.01 96.64 0.00 0.00 0.00 0.00 2167127 307073 vmx-vcpu-19:cen 1 0.01 0.01 0.00 99.83 4.30 0.01 95.53 0.00 0.00 0.00 0.00 2167128 307073 vmx-vcpu-20:cen 1 0.02 0.01 0.00 99.83 14.35 0.00 85.48 0.00 0.00 0.00 0.00 2167129 307073 vmx-vcpu-21:cen 1 0.01 0.01 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 2167130 307073 vmx-vcpu-22:cen 1 0.17 0.14 0.00 99.70 1.55 0.01 98.15 0.01 0.00 0.00 0.00 2167131 307073 vmx-vcpu-23:cen 1 0.00 0.00 0.00 99.84 25.39 0.00 74.45 0.00 0.00 0.00 0.00 <snip> |
1. taskset
taskset 명령어는 running 중인 process의 CPU affinity를 확인하거나 설정하는 도구
특정 process의 affinity mask를 확인하는 방법
| [root@localhost etc]# ps -ef | grep mysqld mysql 2237 1 0 05:48 ? 00:00:01 /usr/sbin/mysqld root 2333 2022 0 06:02 pts/0 00:00:00 grep --color=auto mysqld [root@localhost etc]# taskset -p 2237 pid 2237's current affinity mask: ffffffff |
affinity mask는 2진수 값으로 설정하며 이를 16진수로 표현
| 8개의 논리 CPU에 대한 마스크는 8자리 00000000 첫 번째와 두 번째 CPU 00000011 => 0x03 첫 번째와 마지막 CPU 10000001 => 0x81 모든 홀수 번째 CPU 01010101 => 0x55 모든 CPU 11111111 => 0xff |
2. numactl
NUMA 정책을 설정하는 도구
현재 프로세스의 NUMA 정책 확인
| [root@localhost ~]# numactl --show policy: default preferred node: current physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 cpubind: 0 1 nodebind: 0 1 membind: 0 1 |
현재 사용 가능한 NUMA 정보 확인
| [root@localhost ~]# numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 node 0 size: 96528 MB node 0 free: 95032 MB node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 node 1 size: 96749 MB node 1 free: 95442 MB node distances: node 0 1 0: 10 20 1: 20 10 |
NUMA 정책
| Policy settings are: --all, -a Unset default cpuset awareness, so user can use all possible CPUs/nodes for following policy settings. --interleave=nodes, -i nodes Set a memory interleave policy. Memory will be allocated using round robin on nodes. When memory can‐ not be allocated on the current interleave target fall back to other nodes. Multiple nodes may be specified on --interleave, --membind and --cpunodebind. --membind=nodes, -m nodes Only allocate memory from nodes. Allocation will fail when there is not enough memory available on these nodes. nodes may be specified as noted above. --cpunodebind=nodes, -N nodes Only execute command on the CPUs of nodes. Note that nodes may consist of multiple CPUs. nodes may be specified as noted above. --physcpubind=cpus, -C cpus Only execute process on cpus. This accepts cpu numbers as shown in the processor fields of /proc/cpuinfo, or relative cpus as in relative to the current cpuset. You may specify "all", which means all cpus in the current cpuset. Physical cpus may be specified as N,N,N or N-N or N,N-N or N- N,N-N and so forth. Relative cpus may be specifed as +N,N,N or +N-N or +N,N-N and so forth. The + indicates that the cpu numbers are relative to the process' set of allowed cpus in its current cpuset. A !N-N notation indicates the inverse of N-N, in other words all cpus except N-N. If used with + notation, specify !+N-N. --localalloc, -l Always allocate on the current node. --preferred=node Preferably allocate memory on node, but if memory cannot be allocated there fall back to other nodes. This option takes only a single node number. Relative notation may be used. --show, -s Show NUMA policy settings of the current process. --hardware, -H Show inventory of available nodes on the system. |
--membind 옵션을 사용하여 프로세스를 구동하면, 지정한 NUMA Node의 메모리가 부족한 경우 프로세스 종료
--preferred 옵션을 사용하여 프로세스를 구동하면, 선호 NUMA Node의 메모리가 부족한 경우 다른 NUMA Node의 메모리 사용
--cpunobind 옵션을 사용하여 프로세스를 구동하면, 가급적 프로세스가 동작하고 있는 CPU의 Local Memory에 접근 시도, 특정 프로세스가 전체 Core를 사용하지 않는 방식이라면 이러한 방법이 좋은 전략이 될 수 있음
--interleave=all 옵션을 사용하여 프로세스를 구동하면, 모든 NUMA Node에 Memory가 골고루 할당, 하나의 NUMA Node가 보유한 Memory 크기 이상을 필요로 하는 Application에 유리한 정책일 수 있음
만약, mysql 프로세스에 정책을 부여하려면 다음과 같이 numactl 사용 가능
# numactl --interleave=all mysqld &
다음과 같은 에러 발생 시, /etc/my.cnf 파일에 user=mysql 라인 추가
| [root@localhost ~]# numactl --interleave=all mysqld 2023-06-09T08:50:15.273310Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.33) starting as process 6532 2023-06-09T08:50:15.275593Z 0 [ERROR] [MY-010123] [Server] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root! 2023-06-09T08:50:15.275639Z 0 [ERROR] [MY-010119] [Server] Aborting 2023-06-09T08:50:15.276252Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.33) MySQL Community Server - GPL. |
interleave=all 옵션을 설정하고 나서, mysqld의 NUMA 상태
## NUMA Node0와 NUMA Node1의 메모리 할당량의 거의 비슷
| [root@localhost ~]# numactl --show policy: default preferred node: current physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 cpubind: 0 1 nodebind: 0 1 membind: 0 1 [root@localhost ~]# numastat -p mysqld Per-node process memory usage (in MBs) for PID 2012 (mysqld) Node 0 Node 1 Total --------------- --------------- --------------- Huge 0.00 0.00 0.00 Heap 18.84 18.84 37.69 Stack 0.03 0.02 0.05 Private 169.85 158.07 327.93 ---------------- --------------- --------------- --------------- Total 188.72 176.94 365.66 [root@localhost ~]# sysbench --mysql-host=localhost --mysql-port=3306 --mysql-user=sysbench --mysql-password=P@ssw0rd --mysql-db=sysbench --threads=20 --report-interval=5 --table-size=5000000 --tables=5 /usr/share/sysbench/oltp_read_write.lua run sysbench 1.0.17 (using system LuaJIT 2.0.4) Running the test with following options: Number of threads: 20 Report intermediate results every 5 second(s) Initializing random number generator from current time Initializing worker threads... Threads started! [ 5s ] thds: 20 tps: 964.48 qps: 19339.37 (r/w/o: 13543.29/3863.11/1932.96) lat (ms,95%): 37.56 err/s: 0.00 reconn/s: 0.00 [ 10s ] thds: 20 tps: 1356.41 qps: 27116.32 (r/w/o: 18982.08/5421.42/2712.81) lat (ms,95%): 23.95 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 162764 write: 46504 other: 23252 total: 232520 transactions: 11626 (1160.65 per sec.) queries: 232520 (23212.92 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 10.0153s total number of events: 11626 Latency (ms): min: 2.76 avg: 17.22 max: 126.06 95th percentile: 32.53 sum: 200159.02 Threads fairness: events (avg/stddev): 581.3000/7.93 execution time (avg/stddev): 10.0080/0.00 [root@localhost ~]# numastat -p mysqld Per-node process memory usage (in MBs) for PID 2012 (mysqld) Node 0 Node 1 Total --------------- --------------- --------------- Huge 0.00 0.00 0.00 Heap 18.88 18.89 37.77 Stack 0.03 0.02 0.05 Private 265.52 251.58 517.10 ---------------- --------------- --------------- --------------- Total 284.43 270.49 554.92 |
MySql의 경우에는 Configuration 파일을 이용하여 NUMA 정책 변경 가능
| [root@localhost ~]# grep leave /etc/my.cnf innodb_numa_interleave = 1 |
위 Configuration 설정 후 MySql 중지
| [root@localhost ~]# mysqladmin -u root -p shutdown Enter password: [root@localhost ~]# systemctl status mysqld ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; disabled; vendor preset: disabled) Active: inactive (dead) Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html |
MySql 시작
| [root@localhost bin]# systemctl start mysqld [root@localhost bin]# systemctl status mysqld ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; disabled; vendor preset: disabled) Active: active (running) since Sat 2023-06-10 08:37:35 EDT; 12s ago Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html Process: 3416 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS) Main PID: 3440 (mysqld) Status: "Server is operational" CGroup: /system.slice/mysqld.service └─3440 /usr/sbin/mysqld Jun 10 08:37:34 localhost.localdomain systemd[1]: Starting MySQL Server... Jun 10 08:37:35 localhost.localdomain systemd[1]: Started MySQL Server. |
/etc/my.cnf 설정 반영 여부 확인
| [root@localhost bin]# mysql -h localhost -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 8.0.33 MySQL Community Server - GPL Copyright (c) 2000, 2023, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show global variables like '%numa%'; +------------------------+-------+ | Variable_name | Value | +------------------------+-------+ | innodb_numa_interleave | ON | +------------------------+-------+ 1 row in set (0.01 sec) |
3. numastat
현재 NUMA 운영 상태 확인 도구
numa_miss, numa_foreign이 높다는 것은 의도한대로 메모리가 할당되지 못하고 있다는 것을 의미하기 때문에 이를 줄이는 것이 보다 효율성을 높이는 방법
## local_node : 프로세스가 local node에서 실행되고, 메모리도 local node에서 할당한 케이스
## other_node : 프로세스가 other node에서 실행되고, 메모리는 local node에서 할당한 케이스
## NUMA Node0의 local_node와 NUMA Node1의 other_node는 NUMA Node0에서 실행 중인 프로세스가 사용 중인 메모리
## NUMA Node1의 local_node와 NUMA Node0의 other_node는 NUMA Node1에서 실행 중인 프로세스가 사용 중인 메모리
| [root@localhost ~]# numastat node0 node1 numa_hit 297420 267900 numa_miss 0 0 numa_foreign 0 0 interleave_hit 58737 58387 local_node 289530 215941 other_node 7890 51959 * numa_hit is memory successfully allocated on this node as intended. * numa_miss is memory allocated on this node despite the process preferring some different node. Each numa_miss has a numa_foreign on another node. * numa_foreign is memory intended for this node, but actually allocated on some different node. Each .I numa_foreign has a .I numa_miss on another node. * interleave_hit is interleaved memory successfully allocated on this node as intended. * interleave_hit is the number of interleave policy allocations that were intended for a specific node and succeeded there. * local_node is memory allocated on this node while a process was running on it. * other_node is memory allocated on this node while a process was running on some other node. |
메모리 정보 확인
어느 NUMA Node에 Active Memory가 더 많은지 확인 필요
Active(anon)이 실제 할당하여 사용하는 메모리
| [root@localhost ~]# numastat -m Per-node system memory usage (in MBs): Node 0 Node 1 Total --------------- --------------- --------------- MemTotal 96528.50 96749.23 193277.73 MemFree 95032.36 95442.29 190474.64 MemUsed 1496.14 1306.94 2803.09 Active 302.68 150.99 453.67 Inactive 62.30 90.82 153.13 Active(anon) 278.82 137.39 416.21 Inactive(anon) 8.28 0.27 8.55 Active(file) 23.86 13.60 37.46 Inactive(file) 54.03 90.55 144.58 Unevictable 0.00 0.00 0.00 Mlocked 0.00 0.00 0.00 Dirty 0.00 0.00 0.00 Writeback 0.00 0.00 0.00 FilePages 86.40 104.57 190.98 Mapped 18.72 22.14 40.86 AnonPages 278.38 137.28 415.66 Shmem 8.52 0.43 8.94 KernelStack 4.14 2.84 6.98 PageTables 2.68 2.75 5.43 NFS_Unstable 0.00 0.00 0.00 Bounce 0.00 0.00 0.00 WritebackTmp 0.00 0.00 0.00 Slab 71.09 46.41 117.50 SReclaimable 14.01 13.11 27.12 SUnreclaim 57.08 33.30 90.38 AnonHugePages 252.00 104.00 356.00 HugePages_Total 0.00 0.00 0.00 HugePages_Free 0.00 0.00 0.00 HugePages_Surp 0.00 0.00 0.00 |
특정 프로세스의 상태 확인
아래 결과는 Node0에 더 많은 Memory를 사용하고 있다는 것을 표현
| [root@localhost ~]# numastat -p mysql Per-node process memory usage (in MBs) for PID 1546 (mysqld) Node 0 Node 1 Total --------------- --------------- --------------- Huge 0.00 0.00 0.00 Heap 33.81 3.95 37.76 Stack 0.02 0.03 0.05 Private 226.56 123.18 349.74 ---------------- --------------- --------------- --------------- Total 260.39 127.16 387.55 |
더 자세한 상태는 /proc/pid/numa_maps 에서 확인 가능
| [root@localhost ~]# cat /proc/2304/numa_maps 00400000 default file=/usr/sbin/mysqld mapped=467 active=1 N1=467 kernelpagesize_kB=4 00c5c000 default file=/usr/sbin/mysqld mapped=2546 active=0 N1=2546 kernelpagesize_kB=4 02ade000 default file=/usr/sbin/mysqld mapped=476 active=0 N1=476 kernelpagesize_kB=4 03c23000 default file=/usr/sbin/mysqld anon=191 dirty=191 mapped=235 active=191 N1=235 kernelpagesize_kB=4 03daf000 default file=/usr/sbin/mysqld anon=68 dirty=68 mapped=163 active=68 N1=163 kernelpagesize_kB=4 0414e000 default anon=810 dirty=1322 active=810 N1=1322 kernelpagesize_kB=4 0601e000 default heap anon=6114 dirty=9698 active=6114 N1=9698 kernelpagesize_kB=4 7f9d00000000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9d00021000 default 7f9d04000000 default anon=7 dirty=7 N1=7 kernelpagesize_kB=4 7f9d04021000 default 7f9d08000000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9d08021000 default 7f9d0c000000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9d0c021000 default 7f9d10000000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9d10021000 default 7f9d14000000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9d14021000 default 7f9d18000000 default anon=383 dirty=383 N1=383 kernelpagesize_kB=4 7f9d18183000 default 7f9d1c000000 default anon=391 dirty=391 N1=391 kernelpagesize_kB=4 7f9d1c18b000 default 7f9d20000000 default anon=354 dirty=354 N1=354 kernelpagesize_kB=4 7f9d20251000 default 7f9d24000000 default anon=366 dirty=366 N1=366 kernelpagesize_kB=4 7f9d24178000 default 7f9d28000000 default anon=23 dirty=8215 active=23 N1=8215 kernelpagesize_kB=4 7f9d2a01b000 default 7f9d2c000000 default anon=356 dirty=868 active=356 N1=868 kernelpagesize_kB=4 7f9d2c455000 default 7f9d30000000 default anon=402 dirty=402 N1=402 kernelpagesize_kB=4 7f9d3019c000 default 7f9d34000000 default anon=356 dirty=356 N1=356 kernelpagesize_kB=4 7f9d34169000 default 7f9d38000000 default anon=397 dirty=397 N1=397 kernelpagesize_kB=4 7f9d38262000 default 7f9d3c000000 default anon=354 dirty=354 N1=354 kernelpagesize_kB=4 7f9d3c16b000 default 7f9d40000000 default anon=369 dirty=369 N1=369 kernelpagesize_kB=4 7f9d40175000 default 7f9d44000000 default anon=398 dirty=398 N1=398 kernelpagesize_kB=4 7f9d44194000 default 7f9d48000000 default anon=369 dirty=369 N1=369 kernelpagesize_kB=4 7f9d48175000 default 7f9d4c000000 default anon=1897 dirty=2921 active=1897 N1=2921 kernelpagesize_kB=4 7f9d4cb6f000 default 7f9d50000000 default anon=372 dirty=372 N1=372 kernelpagesize_kB=4 7f9d5017b000 default 7f9d54000000 default anon=375 dirty=375 N1=375 kernelpagesize_kB=4 7f9d54259000 default 7f9d58000000 default anon=373 dirty=373 N1=373 kernelpagesize_kB=4 7f9d58176000 default 7f9d5c000000 default anon=373 dirty=373 N1=373 kernelpagesize_kB=4 7f9d5c17b000 default 7f9d60000000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9d60021000 default 7f9d64000000 default anon=105 dirty=617 active=105 N1=617 kernelpagesize_kB=4 7f9d64269000 default 7f9d68000000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9d68021000 default 7f9d6c000000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9d6c021000 default 7f9d70000000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9d70021000 default 7f9d74000000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9d74021000 default 7f9d78000000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9d78021000 default 7f9d7c000000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9d7c021000 default 7f9d80000000 default anon=66 dirty=66 N1=66 kernelpagesize_kB=4 7f9d80043000 default 7f9d84000000 default anon=4 dirty=4 N1=4 kernelpagesize_kB=4 7f9d84021000 default 7f9d88000000 default anon=86 dirty=86 N1=86 kernelpagesize_kB=4 7f9d88057000 default 7f9d8c000000 default anon=392 dirty=392 N1=392 kernelpagesize_kB=4 7f9d8c25d000 default 7f9d90000000 default anon=92 dirty=92 N1=92 kernelpagesize_kB=4 7f9d9005d000 default 7f9d94000000 default anon=86 dirty=86 N1=86 kernelpagesize_kB=4 7f9d94057000 default 7f9d98000000 default anon=97 dirty=609 active=97 N1=609 kernelpagesize_kB=4 7f9d98263000 default 7f9d9c000000 default anon=354 dirty=354 N1=354 kernelpagesize_kB=4 7f9d9c16c000 default 7f9da0000000 default anon=12 dirty=12 N1=12 kernelpagesize_kB=4 7f9da0021000 default 7f9da4000000 default anon=13 dirty=13 N1=13 kernelpagesize_kB=4 7f9da4021000 default 7f9da8000000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9da8021000 default 7f9dae5f9000 default 7f9dae5fa000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dae6fb000 default 7f9dae6fc000 default anon=17 dirty=17 N1=17 kernelpagesize_kB=4 7f9dae7fd000 default 7f9dae7fe000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9daeffe000 default 7f9daefff000 default anon=11 dirty=11 N1=11 kernelpagesize_kB=4 7f9daf7ff000 default 7f9daf800000 default dirty=512 active=0 N1=512 kernelpagesize_kB=4 7f9db0000000 default anon=14 dirty=14 N1=14 kernelpagesize_kB=4 7f9db0021000 default 7f9db40eb000 default 7f9db40ec000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db41ed000 default 7f9db41ee000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db42ef000 default 7f9db42f0000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db43f1000 default 7f9db43f2000 default anon=17 dirty=17 N1=17 kernelpagesize_kB=4 7f9db44f3000 default 7f9db44f4000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db45f5000 default 7f9db45f6000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db46f7000 default 7f9db46f8000 default anon=16 dirty=16 N1=16 kernelpagesize_kB=4 7f9db47f9000 default 7f9db47fa000 default anon=10 dirty=10 N1=10 kernelpagesize_kB=4 7f9db4ffa000 default 7f9db4ffb000 default anon=11 dirty=11 N1=11 kernelpagesize_kB=4 7f9db57fb000 default 7f9db57fc000 default anon=7 dirty=7 N1=7 kernelpagesize_kB=4 7f9db5ffc000 default 7f9db5ffd000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9db67fd000 default 7f9db67fe000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9db6ffe000 default 7f9db6fff000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9db77ff000 default 7f9db7800000 default anon=512 dirty=512 N1=512 kernelpagesize_kB=4 7f9db8000000 default anon=4 dirty=4 N1=4 kernelpagesize_kB=4 7f9db8021000 default 7f9dbc008000 default 7f9dbc009000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbc10a000 default 7f9dbc10b000 default anon=16 dirty=16 N1=16 kernelpagesize_kB=4 7f9dbc20c000 default 7f9dbc20d000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbc30e000 default 7f9dbc30f000 default anon=17 dirty=17 N1=17 kernelpagesize_kB=4 7f9dbc410000 default 7f9dbc411000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbc512000 default 7f9dbc513000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbcd13000 default 7f9dbcd14000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbd514000 default 7f9dbd515000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dbdd15000 default 7f9dbdd16000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9dbe516000 default 7f9dbe517000 default anon=11 dirty=11 N1=11 kernelpagesize_kB=4 7f9dbed17000 default 7f9dbed18000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dbf518000 default 7f9dbf519000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dbfd19000 default 7f9dbfd1a000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dc051a000 default 7f9dc051b000 default anon=1309 dirty=8477 active=1309 N1=8477 kernelpagesize_kB=4 7f9dc2d1d000 default 7f9dc2d1e000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9dc351e000 default 7f9dc351f000 default anon=512 dirty=33792 active=512 N1=33792 kernelpagesize_kB=4 7f9dcc000000 default anon=363 dirty=875 active=363 N1=875 kernelpagesize_kB=4 7f9dcc36c000 default 7f9dd00ec000 default 7f9dd00ed000 default anon=16 dirty=16 N1=16 kernelpagesize_kB=4 7f9dd01ee000 default 7f9dd01ef000 default anon=16 dirty=16 N1=16 kernelpagesize_kB=4 7f9dd02f0000 default 7f9dd02f1000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dd03f2000 default 7f9dd03f3000 default anon=6 dirty=6 N1=6 kernelpagesize_kB=4 7f9dd04f4000 default 7f9dd04f5000 default anon=16 dirty=16 N1=16 kernelpagesize_kB=4 7f9dd05f6000 default 7f9dd05f7000 default anon=17 dirty=17 N1=17 kernelpagesize_kB=4 7f9dd06f8000 default 7f9dd06f9000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9dd07f9000 default 7f9dd07fa000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9dd0ffa000 default 7f9dd0ffb000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9dd17fb000 default 7f9dd17fc000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9dd1ffc000 default 7f9dd1ffd000 default anon=8 dirty=8 N1=8 kernelpagesize_kB=4 7f9dd27fd000 default 7f9dd27fe000 default anon=8 dirty=8 N1=8 kernelpagesize_kB=4 7f9dd2ffe000 default 7f9dd2fff000 default anon=8 dirty=8 N1=8 kernelpagesize_kB=4 7f9dd37ff000 default 7f9dd3800000 default anon=512 dirty=512 N1=512 kernelpagesize_kB=4 7f9dd4000000 default anon=3666 dirty=13394 active=3666 N1=13394 kernelpagesize_kB=4 7f9dd7452000 default 7f9dd8025000 default 7f9dd8026000 default anon=9 dirty=9 N1=9 kernelpagesize_kB=4 7f9dd8126000 default 7f9dd8127000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dd8228000 default 7f9dd8229000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9dd832a000 default 7f9dd832b000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9dd842b000 default 7f9dd842c000 default anon=791 dirty=791 N1=791 kernelpagesize_kB=4 7f9dd8c12000 default file=/usr/lib64/mysql/plugin/component_validate_password.so mapped=11 active=1 N1=11 kernelpagesize_kB=4 7f9dd8c1d000 default file=/usr/lib64/mysql/plugin/component_validate_password.so mapped=6 active=0 N1=6 kernelpagesize_kB=4 7f9dd8c29000 default file=/usr/lib64/mysql/plugin/component_validate_password.so mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd8c2e000 default file=/usr/lib64/mysql/plugin/component_validate_password.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dd8c2f000 default file=/usr/lib64/mysql/plugin/component_validate_password.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dd8c30000 default 7f9dd8c31000 default anon=614 dirty=614 N1=614 kernelpagesize_kB=4 7f9dd8f95000 default 7f9dd8f96000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dd9796000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd9799000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd979c000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd979f000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd97a2000 default anon=67 dirty=67 N1=67 kernelpagesize_kB=4 7f9dd97e5000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd97e8000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dd97eb000 default anon=1357 dirty=2893 active=1357 N1=2893 kernelpagesize_kB=4 7f9dda338000 default 7f9dda339000 default anon=1024 dirty=1536 active=1024 N1=1536 kernelpagesize_kB=4 7f9ddb83b000 default file=/usr/lib64/mysql/plugin/component_reference_cache.so mapped=2 active=1 N1=2 kernelpagesize_kB=4 7f9ddb83d000 default file=/usr/lib64/mysql/plugin/component_reference_cache.so mapped=2 active=0 N1=2 kernelpagesize_kB=4 7f9ddb842000 default file=/usr/lib64/mysql/plugin/component_reference_cache.so mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9ddb844000 default file=/usr/lib64/mysql/plugin/component_reference_cache.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9ddb845000 default file=/usr/lib64/mysql/plugin/component_reference_cache.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9ddb846000 default file=/usr/lib64/libnss_files-2.17.so mapped=5 mapmax=21 N1=5 kernelpagesize_kB=4 7f9ddb852000 default file=/usr/lib64/libnss_files-2.17.so 7f9ddba51000 default file=/usr/lib64/libnss_files-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9ddba52000 default file=/usr/lib64/libnss_files-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9ddba53000 default anon=37590 dirty=47830 active=37590 N1=47830 kernelpagesize_kB=4 7f9de752f000 default file=/usr/lib64/libpcre.so.1.2.0 mapped=2 mapmax=23 N1=2 kernelpagesize_kB=4 7f9de758f000 default file=/usr/lib64/libpcre.so.1.2.0 7f9de778f000 default file=/usr/lib64/libpcre.so.1.2.0 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7790000 default file=/usr/lib64/libpcre.so.1.2.0 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7791000 default file=/usr/lib64/libselinux.so.1 mapped=8 mapmax=22 N1=8 kernelpagesize_kB=4 7f9de77b5000 default file=/usr/lib64/libselinux.so.1 7f9de79b4000 default file=/usr/lib64/libselinux.so.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de79b5000 default file=/usr/lib64/libselinux.so.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de79b6000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de79b8000 default file=/usr/lib64/libresolv-2.17.so mapped=4 mapmax=17 N1=4 kernelpagesize_kB=4 7f9de79ce000 default file=/usr/lib64/libresolv-2.17.so 7f9de7bce000 default file=/usr/lib64/libresolv-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7bcf000 default file=/usr/lib64/libresolv-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7bd0000 default 7f9de7bd2000 default file=/usr/lib64/libkeyutils.so.1.5 mapped=2 mapmax=13 N1=2 kernelpagesize_kB=4 7f9de7bd5000 default file=/usr/lib64/libkeyutils.so.1.5 7f9de7dd4000 default file=/usr/lib64/libkeyutils.so.1.5 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7dd5000 default file=/usr/lib64/libkeyutils.so.1.5 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7dd6000 default file=/usr/lib64/libkrb5support.so.0.1 mapped=4 mapmax=13 N1=4 kernelpagesize_kB=4 7f9de7de4000 default file=/usr/lib64/libkrb5support.so.0.1 7f9de7fe4000 default file=/usr/lib64/libkrb5support.so.0.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7fe5000 default file=/usr/lib64/libkrb5support.so.0.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de7fe6000 default file=/usr/lib64/libz.so.1.2.7 mapped=3 mapmax=20 N1=3 kernelpagesize_kB=4 7f9de7ffb000 default file=/usr/lib64/libz.so.1.2.7 7f9de81fa000 default file=/usr/lib64/libz.so.1.2.7 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de81fb000 default file=/usr/lib64/libz.so.1.2.7 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de81fc000 default file=/usr/lib64/libk5crypto.so.3.1 mapped=5 mapmax=13 N1=5 kernelpagesize_kB=4 7f9de822d000 default file=/usr/lib64/libk5crypto.so.3.1 7f9de842c000 default file=/usr/lib64/libk5crypto.so.3.1 anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9de842e000 default file=/usr/lib64/libk5crypto.so.3.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de842f000 default file=/usr/lib64/libcom_err.so.2.1 mapped=2 mapmax=13 N1=2 kernelpagesize_kB=4 7f9de8432000 default file=/usr/lib64/libcom_err.so.2.1 7f9de8631000 default file=/usr/lib64/libcom_err.so.2.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de8632000 default file=/usr/lib64/libcom_err.so.2.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de8633000 default file=/usr/lib64/libkrb5.so.3.3 mapped=37 mapmax=14 N1=37 kernelpagesize_kB=4 7f9de870c000 default file=/usr/lib64/libkrb5.so.3.3 7f9de890b000 default file=/usr/lib64/libkrb5.so.3.3 anon=14 dirty=14 N1=14 kernelpagesize_kB=4 7f9de8919000 default file=/usr/lib64/libkrb5.so.3.3 anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9de891c000 default file=/usr/lib64/libgssapi_krb5.so.2.2 mapped=13 mapmax=14 N1=13 kernelpagesize_kB=4 7f9de8966000 default file=/usr/lib64/libgssapi_krb5.so.2.2 7f9de8b66000 default file=/usr/lib64/libgssapi_krb5.so.2.2 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de8b67000 default file=/usr/lib64/libgssapi_krb5.so.2.2 anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9de8b69000 default file=/usr/lib64/libc-2.17.so mapped=190 mapmax=29 N1=190 kernelpagesize_kB=4 7f9de8d2d000 default file=/usr/lib64/libc-2.17.so 7f9de8f2c000 default file=/usr/lib64/libc-2.17.so anon=4 dirty=4 N1=4 kernelpagesize_kB=4 7f9de8f30000 default file=/usr/lib64/libc-2.17.so anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9de8f32000 default anon=5 dirty=5 N1=5 kernelpagesize_kB=4 7f9de8f37000 default file=/usr/lib64/libgcc_s-4.8.5-20150702.so.1 mapped=11 mapmax=21 N1=11 kernelpagesize_kB=4 7f9de8f4c000 default file=/usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f9de914b000 default file=/usr/lib64/libgcc_s-4.8.5-20150702.so.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de914c000 default file=/usr/lib64/libgcc_s-4.8.5-20150702.so.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de914d000 default file=/usr/lib64/libm-2.17.so mapped=24 mapmax=18 active=21 N1=24 kernelpagesize_kB=4 7f9de924e000 default file=/usr/lib64/libm-2.17.so 7f9de944d000 default file=/usr/lib64/libm-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de944e000 default file=/usr/lib64/libm-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de944f000 default file=/usr/lib64/libstdc++.so.6.0.19 mapped=138 mapmax=5 active=119 N1=138 kernelpagesize_kB=4 7f9de9538000 default file=/usr/lib64/libstdc++.so.6.0.19 7f9de9738000 default file=/usr/lib64/libstdc++.so.6.0.19 anon=8 dirty=8 N1=8 kernelpagesize_kB=4 7f9de9740000 default file=/usr/lib64/libstdc++.so.6.0.19 anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9de9742000 default anon=4 dirty=4 N1=4 kernelpagesize_kB=4 7f9de9757000 default file=/usr/lib64/libdl-2.17.so mapped=2 mapmax=24 N1=2 kernelpagesize_kB=4 7f9de9759000 default file=/usr/lib64/libdl-2.17.so 7f9de9959000 default file=/usr/lib64/libdl-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de995a000 default file=/usr/lib64/libdl-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de995b000 default file=/usr/lib64/libnuma.so.1.0.0 mapped=9 mapmax=2 N1=9 kernelpagesize_kB=4 7f9de9965000 default file=/usr/lib64/libnuma.so.1.0.0 7f9de9b65000 default file=/usr/lib64/libnuma.so.1.0.0 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de9b66000 default file=/usr/lib64/libnuma.so.1.0.0 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de9b67000 default file=/usr/lib64/libaio.so.1.0.1 mapped=1 N1=1 kernelpagesize_kB=4 7f9de9b68000 default file=/usr/lib64/libaio.so.1.0.1 7f9de9d67000 default file=/usr/lib64/libaio.so.1.0.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de9d68000 default file=/usr/lib64/libaio.so.1.0.1 anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9de9d69000 default file=/usr/lib64/libcrypto.so.1.0.2k mapped=250 mapmax=14 active=216 N1=250 kernelpagesize_kB=4 7f9de9fa0000 default file=/usr/lib64/libcrypto.so.1.0.2k 7f9dea19f000 default file=/usr/lib64/libcrypto.so.1.0.2k anon=28 dirty=28 N1=28 kernelpagesize_kB=4 7f9dea1bb000 default file=/usr/lib64/libcrypto.so.1.0.2k anon=13 dirty=13 N1=13 kernelpagesize_kB=4 7f9dea1c8000 default anon=3 dirty=3 N1=3 kernelpagesize_kB=4 7f9dea1cc000 default file=/usr/lib64/libssl.so.1.0.2k mapped=54 mapmax=13 active=36 N1=54 kernelpagesize_kB=4 7f9dea233000 default file=/usr/lib64/libssl.so.1.0.2k 7f9dea433000 default file=/usr/lib64/libssl.so.1.0.2k anon=4 dirty=4 N1=4 kernelpagesize_kB=4 7f9dea437000 default file=/usr/lib64/libssl.so.1.0.2k anon=7 dirty=7 N1=7 kernelpagesize_kB=4 7f9dea43e000 default file=/usr/lib64/librt-2.17.so mapped=4 mapmax=22 N1=4 kernelpagesize_kB=4 7f9dea445000 default file=/usr/lib64/librt-2.17.so 7f9dea644000 default file=/usr/lib64/librt-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dea645000 default file=/usr/lib64/librt-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dea646000 default file=/usr/lib64/libpthread-2.17.so mapped=20 mapmax=26 N1=20 kernelpagesize_kB=4 7f9dea65d000 default file=/usr/lib64/libpthread-2.17.so 7f9dea85c000 default file=/usr/lib64/libpthread-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dea85d000 default file=/usr/lib64/libpthread-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dea85e000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9dea862000 default file=/usr/lib64/ld-2.17.so mapped=30 mapmax=29 N1=30 kernelpagesize_kB=4 7f9dea886000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9dea889000 default anon=332 dirty=332 N1=332 kernelpagesize_kB=4 7f9dea9d5000 default file=/usr/lib64/mysql/private/libprotobuf-lite.so.3.19.4 mapped=39 active=1 N1=39 kernelpagesize_kB=4 7f9deaa08000 default file=/usr/lib64/mysql/private/libprotobuf-lite.so.3.19.4 mapped=7 active=0 N1=7 kernelpagesize_kB=4 7f9deaa5b000 default file=/usr/lib64/mysql/private/libprotobuf-lite.so.3.19.4 7f9deaa75000 default file=/usr/lib64/mysql/private/libprotobuf-lite.so.3.19.4 anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9deaa77000 default file=/usr/lib64/mysql/private/libprotobuf-lite.so.3.19.4 anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9deaa79000 default anon=2 dirty=2 N1=2 kernelpagesize_kB=4 7f9deaa7b000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9deaa7e000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9deaa81000 default file=/[aio]\040(deleted) mapped=1 active=0 N1=1 kernelpagesize_kB=4 7f9deaa82000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9deaa83000 default file=/usr/lib64/ld-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9deaa84000 default file=/usr/lib64/ld-2.17.so anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7f9deaa85000 default anon=1 dirty=1 N1=1 kernelpagesize_kB=4 7ffd04621000 default stack anon=13 dirty=13 N1=13 kernelpagesize_kB=4 7ffd04793000 default |