2013:05:13-06:00:04 routerth kernel: [836192.578238] Loading kernel module for a network device with CAP_SYS_MODULE (deprecated). Use CAP_NET_ADMIN and alias netdev-ppp0 instead. 2013:05:13-06:04:38 routerth kernel: [836466.806109] afcd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 2013:05:13-06:04:38 routerth kernel: [836466.806120] Pid: 5027, comm: afcd Tainted: G O 3.8.6-4.g0a2a8fa-smp64 #1 2013:05:13-06:04:38 routerth kernel: [836466.806124] Call Trace: 2013:05:13-06:04:38 routerth kernel: [836466.806140] [] ? dump_header+0x68/0x192 2013:05:13-06:04:38 routerth kernel: [836466.806149] [] ? delayacct_end+0x74/0x7f 2013:05:13-06:04:38 routerth kernel: [836466.806157] [] ? get_page_from_freelist+0x68f/0x6e9 2013:05:13-06:04:38 routerth kernel: [836466.806165] [] ? oom_kill_process+0x74/0x2ff 2013:05:13-06:04:38 routerth kernel: [836466.806173] [] ? oom_badness+0x94/0xf4 2013:05:13-06:04:38 routerth kernel: [836466.806181] [] ? out_of_memory+0x307/0x33a 2013:05:13-06:04:38 routerth kernel: [836466.806188] [] ? __alloc_pages_nodemask+0x4cf/0x58a 2013:05:13-06:04:38 routerth kernel: [836466.806197] [] ? filemap_fault+0x23e/0x32b 2013:05:13-06:04:38 routerth kernel: [836466.806205] [] ? __do_fault+0xa6/0x3c2 2013:05:13-06:04:38 routerth kernel: [836466.806213] [] ? handle_pte_fault+0x30a/0x7b0 2013:05:13-06:04:38 routerth kernel: [836466.806221] [] ? lock_hrtimer_base+0x1b/0x3c 2013:05:13-06:04:38 routerth kernel: [836466.806228] [] ? hrtimer_try_to_cancel+0x62/0x6b 2013:05:13-06:04:38 routerth kernel: [836466.806237] [] ? do_page_fault+0x416/0x43a 2013:05:13-06:04:38 routerth kernel: [836466.806246] [] ? sys_epoll_wait+0x334/0x3a0 2013:05:13-06:04:38 routerth kernel: [836466.806254] [] ? try_to_wake_up+0x19c/0x19c 2013:05:13-06:04:38 routerth kernel: [836466.806262] [] ? timekeeping_get_ns+0x12/0x35 2013:05:13-06:04:38 routerth kernel: [836466.806270] [] ? page_fault+0x28/0x30 2013:05:13-06:04:38 routerth kernel: [836466.806274] Mem-Info: 2013:05:13-06:04:38 routerth kernel: [836466.806279] Node 0 DMA per-cpu: 2013:05:13-06:04:38 routerth kernel: [836466.806285] CPU 0: hi: 0, btch: 1 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806290] CPU 1: hi: 0, btch: 1 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806295] CPU 2: hi: 0, btch: 1 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806299] CPU 3: hi: 0, btch: 1 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806303] Node 0 DMA32 per-cpu: 2013:05:13-06:04:38 routerth kernel: [836466.806308] CPU 0: hi: 186, btch: 31 usd: 30 2013:05:13-06:04:38 routerth kernel: [836466.806313] CPU 1: hi: 186, btch: 31 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806318] CPU 2: hi: 186, btch: 31 usd: 30 2013:05:13-06:04:38 routerth kernel: [836466.806322] CPU 3: hi: 186, btch: 31 usd: 83 2013:05:13-06:04:38 routerth kernel: [836466.806326] Node 0 Normal per-cpu: 2013:05:13-06:04:38 routerth kernel: [836466.806331] CPU 0: hi: 186, btch: 31 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806336] CPU 1: hi: 186, btch: 31 usd: 0 2013:05:13-06:04:38 routerth kernel: [836466.806341] CPU 2: hi: 186, btch: 31 usd: 30 2013:05:13-06:04:38 routerth kernel: [836466.806345] CPU 3: hi: 186, btch: 31 usd: 94 2013:05:13-06:04:38 routerth kernel: [836466.806357] active_anon:713714 inactive_anon:263566 isolated_anon:0 2013:05:13-06:04:38 routerth kernel: [836466.806357] active_file:130 inactive_file:2814 isolated_file:32 2013:05:13-06:04:38 routerth kernel: [836466.806357] unevictable:0 dirty:0 writeback:0 unstable:0 2013:05:13-06:04:38 routerth kernel: [836466.806357] free:6977 slab_reclaimable:4417 slab_unreclaimable:4575 2013:05:13-06:04:38 routerth kernel: [836466.806357] mapped:7048 shmem:8463 pagetables:3410 bounce:0 2013:05:13-06:04:38 routerth kernel: [836466.806357] free_cma:0 2013:05:13-06:04:38 routerth kernel: [836466.806370] Node 0 DMA free:15900kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15652kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes 2013:05:13-06:04:38 routerth kernel: [836466.806386] lowmem_reserve[]: 0 2997 4005 4005 2013:05:13-06:04:38 routerth kernel: [836466.806394] Node 0 DMA32 free:9996kB min:6048kB low:7560kB high:9072kB active_anon:2401164kB inactive_anon:600420kB active_file:88kB inactive_file:7256kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:3069924kB managed:3043408kB mlocked:0kB dirty:0kB writeback:0kB mapped:22640kB shmem:28188kB slab_reclaimable:5956kB slab_unreclaimable:3648kB kernel_stack:416kB pagetables:7260kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:12018 all_unreclaimable? yes 2013:05:13-06:04:38 routerth kernel: [836466.806411] lowmem_reserve[]: 0 0 1008 1008 2013:05:13-06:04:38 routerth kernel: [836466.806418] Node 0 Normal free:2012kB min:2032kB low:2540kB high:3048kB active_anon:453692kB inactive_anon:453844kB active_file:432kB inactive_file:4000kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1032192kB managed:982048kB mlocked:0kB dirty:0kB writeback:0kB mapped:5552kB shmem:5664kB slab_reclaimable:11712kB slab_unreclaimable:14644kB kernel_stack:2208kB pagetables:6380kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:7115 all_unreclaimable? yes 2013:05:13-06:04:38 routerth kernel: [836466.806434] lowmem_reserve[]: 0 0 0 0 2013:05:13-06:04:38 routerth kernel: [836466.806441] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15900kB 2013:05:13-06:04:38 routerth kernel: [836466.806469] Node 0 DMA32: 304*4kB (UE) 235*8kB (UEM) 53*16kB (UE) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB (R) 1*4096kB (R) = 10088kB 2013:05:13-06:04:38 routerth kernel: [836466.806493] Node 0 Normal: 253*4kB (UER) 5*8kB (R) 8*16kB (R) 13*32kB (R) 4*64kB (R) 2*128kB (R) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2108kB 2013:05:13-06:04:38 routerth kernel: [836466.806518] 17721 total pagecache pages 2013:05:13-06:04:38 routerth kernel: [836466.806522] 6249 pages in swap cache 2013:05:13-06:04:38 routerth kernel: [836466.806527] Swap cache stats: add 967145, delete 960896, find 77625888/77664021 2013:05:13-06:04:38 routerth kernel: [836466.806531] Free swap = 0kB 2013:05:13-06:04:38 routerth kernel: [836466.806535] Total swap = 1048572kB 2013:05:13-06:04:38 routerth kernel: [836466.845209] 1048560 pages RAM 2013:05:13-06:04:38 routerth kernel: [836466.845219] 37172 pages reserved 2013:05:13-06:04:38 routerth kernel: [836466.845224] 1065040 pages shared 2013:05:13-06:04:38 routerth kernel: [836466.845229] 991913 pages non-shared 2013:05:13-06:04:38 routerth kernel: [836466.845235] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name 2013:05:13-06:04:38 routerth kernel: [836466.845267] [ 2273] 0 2273 592 26 5 59 -1000 udevd 2013:05:13-06:04:38 routerth kernel: [836466.845282] [ 2699] 0 2699 1435 2 6 69 0 syslog-ng 2013:05:13-06:04:38 routerth kernel: [836466.845294] [ 2700] 0 2700 1892 402 7 86 0 syslog-ng 2013:05:13-06:04:38 routerth kernel: [836466.845305] [ 2716] 0 2716 476 2 5 22 0 acpid 2013:05:13-06:04:38 routerth kernel: [836466.845315] [ 2731] 200 2731 648 20 4 26 0 dbus-daemon 2013:05:13-06:04:38 routerth kernel: [836466.845326] [ 2961] 201 2961 1509 126 6 37 0 hald 2013:05:13-06:04:38 routerth kernel: [836466.845337] [ 2962] 0 2962 916 19 5 31 0 hald-runner 2013:05:13-06:04:38 routerth kernel: [836466.845347] [ 2983] 0 2983 1407 0 6 59 0 hald-addon-inpu 2013:05:13-06:04:38 routerth kernel: [836466.845357] [ 2991] 0 2991 1400 23 6 36 0 hald-addon-stor 2013:05:13-06:04:38 routerth kernel: [836466.845367] [ 3001] 201 3001 1331 0 6 58 0 hald-addon-acpi 2013:05:13-06:04:38 routerth kernel: [836466.845377] [ 3020] 0 3020 1056 43 5 152 0 irqd 2013:05:13-06:04:38 routerth kernel: [836466.845387] [ 3040] 0 3040 1559 1039 6 14 0 haveged 2013:05:13-06:04:38 routerth kernel: [836466.845397] [ 3084] 0 3084 11133 2363 24 4660 0 confd.plx 2013:05:13-06:04:38 routerth kernel: [836466.845407] [ 3085] 0 3085 473 1 5 18 0 logger 2013:05:13-06:04:38 routerth kernel: [836466.845449] [ 3101] 0 3101 473 0 5 19 0 confd-queuer 2013:05:13-06:04:38 routerth kernel: [836466.845461] [ 3116] 0 3116 2013 547 8 454 0 confd-qrunner.p 2013:05:13-06:04:38 routerth kernel: [836466.845472] [ 3135] 0 3135 1949 338 8 564 0 sysmond 2013:05:13-06:04:38 routerth kernel: [836466.845483] [ 3246] 0 3246 3893 644 11 1864 0 aua.bin 2013:05:13-06:04:38 routerth kernel: [836466.845494] [ 3247] 0 3247 473 0 5 15 0 logger 2013:05:13-06:04:38 routerth kernel: [836466.845505] [ 3283] 0 3283 11095 1768 23 5220 0 confd.plx 2013:05:13-06:04:38 routerth kernel: [836466.845516] [ 3434] 0 3434 3470 33 11 2042 0 notifier.plx 2013:05:13-06:04:38 routerth kernel: [836466.845527] [ 3457] 880 3457 27387 39 22 114 0 rrdcached 2013:05:13-06:04:38 routerth kernel: [836466.845537] [ 3491] 0 3491 587 0 4 38 0 atd 2013:05:13-06:04:38 routerth kernel: [836466.845548] [ 3620] 999 3620 143423 245 28 43 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845559] [ 3622] 999 3622 143543 3551 31 67 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845569] [ 3623] 999 3623 143488 37 15 52 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845579] [ 3624] 999 3624 143488 3789 19 48 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845589] [ 3625] 999 3625 143701 219 17 92 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845600] [ 3626] 999 3626 1979 44 7 44 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845611] [ 3627] 999 3627 2083 102 7 55 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845622] [ 3683] 0 3683 21745 17550 47 2537 0 mdw.plx 2013:05:13-06:04:38 routerth kernel: [836466.845633] [ 3726] 0 3726 2893 677 9 1141 0 selfmonng.plx 2013:05:13-06:04:38 routerth kernel: [836466.845644] [ 3730] 0 3730 473 1 5 18 0 logger 2013:05:13-06:04:38 routerth kernel: [836466.845655] [ 3738] 0 3738 472 10 3 8 0 daemon-watcher 2013:05:13-06:04:38 routerth kernel: [836466.845666] [ 3739] 0 3739 604 2 5 33 0 mingetty 2013:05:13-06:04:38 routerth kernel: [836466.845676] [ 3740] 0 3740 604 2 5 32 0 mingetty 2013:05:13-06:04:38 routerth kernel: [836466.845686] [ 3741] 0 3741 604 2 5 29 0 mingetty 2013:05:13-06:04:38 routerth kernel: [836466.845695] [ 3742] 0 3742 604 2 5 33 0 mingetty 2013:05:13-06:04:38 routerth kernel: [836466.845703] [ 3744] 0 3744 2823 129 9 1616 0 selfmonng.plx 2013:05:13-06:04:38 routerth kernel: [836466.845712] [ 4017] 0 4017 736 23 5 19 0 DSL.sh 2013:05:13-06:04:38 routerth kernel: [836466.845721] [ 4101] 0 4101 3036 1141 10 800 0 dns-resolver.pl 2013:05:13-06:04:38 routerth kernel: [836466.845731] [ 4177] 0 4177 592 29 4 11 0 cron 2013:05:13-06:04:38 routerth kernel: [836466.845750] [ 4305] 0 4305 2690 42 8 317 0 httpd 2013:05:13-06:04:38 routerth kernel: [836466.845759] [ 4307] 0 4307 473 13 5 5 0 logger 2013:05:13-06:04:38 routerth kernel: [836466.845768] [ 4359] 30 4359 2673 34 8 330 0 httpd 2013:05:13-06:04:38 routerth kernel: [836466.845776] [ 4361] 0 4361 7720 147 9 217 0 ulogd 2013:05:13-06:04:38 routerth kernel: [836466.845785] [ 4366] 999 4366 144243 4857 34 178 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845792] [ 4415] 840 4415 572 0 5 45 0 frox 2013:05:13-06:04:38 routerth kernel: [836466.845800] [ 4524] 0 4524 7813 1283 17 3800 0 smtpd.bin 2013:05:13-06:04:38 routerth kernel: [836466.845808] [ 4539] 0 4539 7780 840 17 4178 0 smtpd.bin 2013:05:13-06:04:38 routerth kernel: [836466.845816] [ 4555] 830 4555 2108 123 7 0 0 exim 2013:05:13-06:04:38 routerth kernel: [836466.845824] [ 4556] 999 4556 144111 465 28 138 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845832] [ 4868] 0 4868 92482 80147 181 673 0 cssd 2013:05:13-06:04:38 routerth kernel: [836466.845839] [ 5027] 888 5027 11299 564 17 1180 0 afcd 2013:05:13-06:04:38 routerth kernel: [836466.845847] [ 5149] 0 5149 3554 833 10 1055 0 ddclient 2013:05:13-06:04:38 routerth kernel: [836466.845855] [ 5177] 0 5177 6749 2870 16 1766 0 epp_client.plx 2013:05:13-06:04:38 routerth kernel: [836466.845863] [12426] 0 12426 477 5 5 14 0 ipv6_watchdog 2013:05:13-06:04:38 routerth kernel: [836466.845871] [12998] 0 12998 2711 791 9 416 0 named 2013:05:13-06:04:38 routerth kernel: [836466.845879] [13105] 0 13105 1229 1 5 78 -1000 sshd 2013:05:13-06:04:38 routerth kernel: [836466.845887] [13110] 0 13110 801 25 5 29 0 ntpd 2013:05:13-06:04:38 routerth kernel: [836466.845895] [13179] 0 13179 1152 124 6 342 0 dhcpd 2013:05:13-06:04:38 routerth kernel: [836466.845904] [13396] 0 13396 1549 17 7 72 0 master 2013:05:13-06:04:38 routerth kernel: [836466.845912] [13398] 51 13398 1578 9 5 85 0 qmgr 2013:05:13-06:04:38 routerth kernel: [836466.845920] [ 2776] 0 2776 1033 8 8 49 0 ctasd 2013:05:13-06:04:38 routerth kernel: [836466.845928] [ 2778] 0 2778 10385 2796 31 1566 0 ctasd 2013:05:13-06:04:38 routerth kernel: [836466.845937] [ 2782] 0 2782 5770 342 18 554 0 ctipd.bin 2013:05:13-06:04:38 routerth kernel: [836466.845944] [ 2843] 0 2843 2194 17 8 92 0 pop3proxy 2013:05:13-06:04:38 routerth kernel: [836466.845952] [ 2844] 999 2844 144088 266 21 174 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.845961] [27741] 810 27741 548571 118834 480 49554 0 httpproxy 2013:05:13-06:04:38 routerth kernel: [836466.845970] [30667] 800 30667 558603 321989 990 173358 0 snort_inline 2013:05:13-06:04:38 routerth kernel: [836466.845978] [ 6684] 30 6684 2783 172 8 310 0 httpd 2013:05:13-06:04:38 routerth kernel: [836466.845986] [11781] 30 11781 2779 245 8 232 0 httpd 2013:05:13-06:04:38 routerth kernel: [836466.845995] [28217] 0 28217 2808 1068 9 532 0 pfilter-reporte 2013:05:13-06:04:38 routerth kernel: [836466.846003] [28218] 0 28218 2844 1146 9 470 0 admin-reporter. 2013:05:13-06:04:38 routerth kernel: [836466.846011] [28219] 0 28219 2823 1150 9 472 0 vpn-reporter.pl 2013:05:13-06:04:38 routerth kernel: [836466.846019] [28220] 0 28220 2605 984 8 416 0 mailsec-reporte 2013:05:13-06:04:38 routerth kernel: [836466.846027] [28221] 0 28221 2808 1221 9 369 0 ips-reporter.pl 2013:05:13-06:04:38 routerth kernel: [836466.846035] [28222] 0 28222 11535 38 13 49 0 websec-reporter 2013:05:13-06:04:38 routerth kernel: [836466.846043] [28223] 0 28223 11250 24 9 50 0 websec-reporter 2013:05:13-06:04:38 routerth kernel: [836466.846051] [28224] 0 28224 2747 1122 8 402 0 waf-reporter.pl 2013:05:13-06:04:38 routerth kernel: [836466.846059] [28228] 999 28228 144470 981 29 204 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.846067] [28229] 999 28229 144089 204 20 217 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.846075] [30280] 999 30280 144117 675 23 18 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.846083] [29568] 999 29568 144116 632 25 18 0 postgres 2013:05:13-06:04:38 routerth kernel: [836466.846091] [ 9726] 51 9726 1564 82 7 0 0 pickup 2013:05:13-06:04:38 routerth kernel: [836466.846099] [ 9995] 0 9995 698 45 5 0 0 pppd-pppoe 2013:05:13-06:04:38 routerth kernel: [836466.846107] [10077] 0 10077 271302 207498 420 0 0 snort_inline 2013:05:13-06:04:38 routerth kernel: [836466.846115] [10409] 0 10409 737 47 5 0 0 snort 2013:05:13-06:04:38 routerth kernel: [836466.846123] [10420] 0 10420 251507 187657 382 0 0 snort_inline 2013:05:13-06:04:38 routerth kernel: [836466.846131] [10448] 0 10448 591 27 5 57 -1000 udevd 2013:05:13-06:04:38 routerth kernel: [836466.846139] [10451] 0 10451 591 27 5 57 -1000 udevd 2013:05:13-06:04:38 routerth kernel: [836466.846148] [11017] 0 11017 476 12 4 0 0 sleep 2013:05:13-06:04:38 routerth kernel: [836466.846154] Out of memory: Kill process 30667 (snort_inline) score 389 or sacrifice child 2013:05:13-06:04:38 routerth kernel: [836466.846160] Killed process 30667 (snort_inline) total-vm:2234412kB, anon-rss:1287956kB, file-rss:0kB