Linux内存爆浆(内存不足)系统日志实例 关于一些刚入门Linux不久又在玩VPS的同学，有时候自己内存爆掉了搞不清东南西北然后到处求救抱怨甚至开喷的情形，今天刚好有空就在dmesg里面抓了一段日志上来做例子。以下是日志内容active_anon:113958 inactive_anon:114006 isolated_anon:32 active_file:6 inactive_file:0 isolated_file:0 unevictable:0 dirty:0 writeback:0 unstable:0 free:1990 slab_reclaimable:2356 slab_unreclaimable:11207 mapped:116 shmem:87 pagetables:7163 bounce:0 Node 0 DMA free:4020kB min:52kB low:64kB high:76kB active_anon:4336kB inactive_anon:4500kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:13856kB mlocked:0kB dirty:0kB writeback:0kB mapped:16kB shmem:0kB slab_reclaimable:60kB slab_unreclaimable:468kB kernel_stack:144kB pagetables:36kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 994 994 994 Node 0 DMA32 free:3940kB min:4004kB low:5004kB high:6004kB active_anon:451496kB inactive_anon:451524kB active_file:36kB inactive_file:0kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:1018080kB mlocked:0kB dirty:0kB writeback:0kB mapped:448kB shmem:348kB slab_reclaimable:9364kB slab_unreclaimable:44360kB kernel_stack:1416kB pagetables:28616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:256 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 Node 0 DMA: 7*4kB 3*8kB 4*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4020kB Node 0 DMA32: 981*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3940kB 432 total pagecache pages 290 pages in swap cache Swap cache stats: add 471821, delete 471531, find 1381770/1427029 Free swap = 0kB Total swap = 524280kB 262143 pages RAM 7859 pages reserved 28661 pages shared 243982 pages non-shared Out of memory: kill process 793 (mysqld_safe) score 91618 or a child Killed process 1047 (mysqld) vsz:1890740kB, anon-rss:26400kB, file-rss:4kB nginx invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0 nginx cpuset=/ mems_allowed=0 Pid: 947, comm: nginx Not tainted 2.6.32-71.29.1.el6.x86_64 #1 Call Trace: [] ? cpuset_print_task_mems_allowed+0x91/0xb0 [] oom_kill_process+0xcb/0x2e0 [] ? select_bad_process+0xd0/0x110 [] __out_of_memory+0x58/0xc0 [] out_of_memory+0x199/0x210 [] __alloc_pages_nodemask+0x832/0x850 [] alloc_pages_current+0x9a/0x100 [] __page_cache_alloc+0x87/0x90 [] filemap_fault+0x1a9/0x510 [] __do_fault+0x54/0x500 [] handle_pte_fault+0xf7/0xad0 [] ? xen_write_msr_safe+0x6b/0x90 [] ? __switch_to+0x1ac/0x320 [] ? __raw_callee_save_xen_pmd_val+0x11/0x1e [] handle_mm_fault+0x1ed/0x2b0 [] ? thread_return+0x4e/0x778 [] do_page_fault+0x123/0x3a0 [] page_fault+0x25/0x30测试机环境为 虚拟化:XEN 物理内存:1G swap:512M 系统为:Centos6 X64 内核版本:2.6.32-71.29.1 应用软件：nginx apache php-fpm mysql php5.3Swap cache stats: add 471821, delete 471531, find 1381770/1427029 Free swap = 0kB Total swap = 524280kB以上为节选内存不足的特征日志 可以看到明显Free swap = 0kB（可用swap缓存为0），Total swap = 524280kB（swap容量512M），通常情况下出现类似的情况就是内存不足了。其他片段日志分析Out of memory: kill process 793 (mysqld_safe) score 91618 or a child Killed process 1047 (mysqld) vsz:1890740kB, anon-rss:26400kB, file-rss:4kB由于没有了内存和swap应用还在请求资源，系统就开始杀掉部分进程来缓解。swap正常情况下是存储一些使用较少的数据，当swap占满后将严重殃及硬盘的IO速度从而会造成例如整机变得很慢很卡的麻烦， 如果是VPS类的在应用优化上能处理解决最好早点解决不然有可能整台服务器都被拖累导致服务商下狠手轻则警告重则撵人给轰出来，或者实在不行就增加内存吧。 一般能迅速占满内存和swap在网站环境内基本是mysql和apache这两个家伙，所以切记要盯紧你的系统资源开销。 更多解决内存不足的方法手段这里就不多讲了可以随意抓几个Linux运维请教。。。。PS.关于Openvz 老版本内核没有Vswap，遇上内存不足基本是直接宕掉，新版内核有Vswap但是具体效果还是没有多大用处（对于vSwap本人未作实战分析）。