1、通过top 查看具体是哪个进程占用内存较多
Tasks:65 total,1 running,64 sleeping,0 stopped,0 zombie%Cpu(s):2.0 us,1.0 sy,0.0 ni,96.3 id,0.3 wa,0.0 hi,0.3 si,0.0 stKiBMem:1016168 total,63544 free,824060 used,128564 buff/cacheKiBSwap:2047996 total,1612356 free,435640 used.49916 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND25894www20030803047570363252 S 3.374.525:55.03 java1 root 2004330021801340 S 0.00.23:20.35 systemd2 root 200000 S 0.00.00:00.43 kthreadd3 root 200000 S 0.00.012:44.14ksoftirqd/05 root 0–20000 S 0.00.00:00.00 kworker/0:0H7 root rt 0000 S 0.00.00:00.00 migration/0
2、定位线程问题(通过命令查看PID 为25894 进程的线程情况),命令:# ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd
# ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd%CPU %MEM PID TID TIME TTY CMD
0.0 74.4 25894 25894 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25896 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.1 74.4 25894 25897 00:01:09 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25898 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25899 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25900 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25901 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25902 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.1 74.4 25894 25903 00:01:18 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
0.0 74.4 25894 25904 00:00:14 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap
……………………………….以下部分省略……………………………….# ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd |wc -l
101
由此可以看到这个PID:25894的进程产生了很多线程。接下来就可以通过jstack查看内存使用的堆栈,具体可以执行 ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd |wc -l 来查具体有多少线程.
3、将PID为25894的堆栈信息打印到jstack.log中,命令: #jstack -l 25894 > jstack.log
4、查看日志,有很多WAITING & TIMED_WAITING 状态的日志信息,具体需要研发查看一下代码的情况;
#cat jstack.log |grep -E WAITING|TIMED_WAITING
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (on object monitor)
java.lang.Thread.State: TIMED_WAITING (on object monitor)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: WAITING (parking)
java.lang.Thread.State: WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (on object monitor)
java.lang.Thread.State: WAITING (on object monitor)
java.lang.Thread.State: WAITING (on object monitor)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
java.lang.Thread.State: TIMED_WAITING (parking)
后经研发检查代码并升级程序至新版本,问题解决。
参考链接:https://www.javatang.com/archives/2017/10/25/36441958.html