Yesterday evening, around 6pm, the memory use on my new ASG 8.1 box started to increase steadily, eventually consuming all of the swap space and most of the RAM. This happened at a time when there were no live users on the LAN, the only activity being my bit torrent traffic, which involves seeding torrents at a steady outbound rate limited to 25 KBytes/sec, and participating in two DHT clouds on the net. I have IDS exceptions defined for that traffic.
I rebooted the ASG router, which brought the memory utilization back down, but I am very puzzled as to what could have caused this.
My webadmin page does not time out and lock up reliably. Last Friday night, when the memory consumption kept increasing, the console was sitting unlocked all night displaying the Dashboard page. The memory consumption was on a steady increase again yesterday, for the same reason. When I manually logged off the webadmin late last night, just before going to bed late, at around 2 am, the memory usage immediately dropped.
When I looked at the memory usage chart this morning, and saw what had happened, I started wondering if it was the system's usage of several hundred MB of swap that kept the webadmin page from timing out and locking. So at 9 am this morning, I took my ASG box down, and added another 512 MB RAM to it. It now has 1.5 GB.
But the webadmin page still does not time out and lock automatically, and when it sits open on a browser tab (on my primary Windows workstatation, I run Firefox with anywhere from 8 to 15 tabs open) the memory usage continues to grow over time.
Take a look at the chart:
The significant memory drops at 2 am and at 2:45 pm happened just after I manually logged off the webadmin page. When the webadmin page is not logged into, the memory usage drops and flat-lines.
I see the same problem with the 7.509 version. It is directly related to having the WebAdmin dashboard up and running. I have been having this problem since week 12, but did not mind. Did a reset now and then when it showed up.
Then today I saw the process it was that used the memory, and tried logging of webadmin. Problem was solved when I logged back on 10 seconds later. (From 80% ram usage to 22% of 4GB.)
Seems there is a memory leak, and that it has been present since about week 12.
The attached file shows the memory consumption with high growth rates after the Upgrade to V8 in September 2011. Even after Upgrade to 8.300 a few weeks ago, the growth rate is still critical.
The memory is always consumed by the snort_inline process.
Our Astaro support at the vendor site (standard support) says that is normal, which I do not believe.
The attached file shows the memory consumption with high growth rates after the Upgrade to V8 in September 2011. Even after Upgrade to 8.300 a few weeks ago, the growth rate is still critical.
The memory is always consumed by the snort_inline process.
Our Astaro support at the vendor site (standard support) says that is normal, which I do not believe.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5942 snort 19 -1 774m 388m 1064 S 2 19.3 279:33.53 snort_inline
5236 postgres 20 0 80188 65m 65m S 0 3.3 3:06.84 postgres
6017 postgres 20 0 82936 62m 61m S 0 3.1 50:06.78 postgres
32753 wwwrun 20 0 62956 59m 8512 S 0 3.0 0:03.99 index.plx
418 root 20 0 53100 24m 2740 S 0 1.2 0:01.36 confd.plx
5292 root 20 0 63816 16m 1736 S 0 0.8 0:17.71 mdw.plx
As i mentioned earlier...with your being a high packet environment especially with p2p this demand a ton of attention and resources by the snort processes. Combine this with the inherent memory increase required to move so many packets and the increased system requirements of 8.3 and your ram usage is actually quite what is expected. I'm no fan of Astaro's decisions most times but this time I think they are correct. Get 3 gig of ram into the box pronto(4 if you can reinstall to 64-bit). I can see you have 1 gig cached which is the aggressive swapping Astaro likes to do. In your case do NOT apply my swappiness=0 trick because you will run out of physical ram with all the ips and packet moving you are doing. You are going to have high ram usage in your scenario especially if you like to leave webadmin open for extended periods of time AND with a high-packet environment w/ips.
The attached file shows the memory consumption with high growth rates after the Upgrade to V8 in September 2011. Even after Upgrade to 8.300 a few weeks ago, the growth rate is still critical.
The memory is always consumed by the snort_inline process.
Our Astaro support at the vendor site (standard support) says that is normal, which I do not believe.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5942 snort 19 -1 774m 388m 1064 S 2 19.3 279:33.53 snort_inline
5236 postgres 20 0 80188 65m 65m S 0 3.3 3:06.84 postgres
6017 postgres 20 0 82936 62m 61m S 0 3.1 50:06.78 postgres
32753 wwwrun 20 0 62956 59m 8512 S 0 3.0 0:03.99 index.plx
418 root 20 0 53100 24m 2740 S 0 1.2 0:01.36 confd.plx
5292 root 20 0 63816 16m 1736 S 0 0.8 0:17.71 mdw.plx
As i mentioned earlier...with your being a high packet environment especially with p2p this demand a ton of attention and resources by the snort processes. Combine this with the inherent memory increase required to move so many packets and the increased system requirements of 8.3 and your ram usage is actually quite what is expected. I'm no fan of Astaro's decisions most times but this time I think they are correct. Get 3 gig of ram into the box pronto(4 if you can reinstall to 64-bit). I can see you have 1 gig cached which is the aggressive swapping Astaro likes to do. In your case do NOT apply my swappiness=0 trick because you will run out of physical ram with all the ips and packet moving you are doing. You are going to have high ram usage in your scenario especially if you like to leave webadmin open for extended periods of time AND with a high-packet environment w/ips.
