Did the limits for RAM on Home edition get changed with SFOS 16.05.2 MR2? I'm only seeing 4GB after update. Still only showing 4GB after rolling back to SFOS 16.05.1 MR-1. I was seeing 6GB usable before upgrade.
Hi All,
Investigating it further, the issue might be associated with the Kernel which is just identifying the RAM on 1 slot out of 2. For example ( 4 GB + 4 GB). Kernel only limits the RAM on single slot
discarding 2nd slot completely so when 6GB limit is imposed only 4GB RAM becomes visible. We are Investigating further.
Thanks for the patience
What is frustrating about the whole thing is that
XG is already getting a black eye from most commercial users that regret the fact that they chose it over UTM. Now you guys are alienating the advanced home user base. Don't get me wrong, I appreciate the very generous hardware quota for home installations but with so many things that break with every release as evident from the posts in this form, I am sure the mystery 8GB user on home license was a not a top problem on Sophos' list.
I agree careful what you say, I am also an enterprise user (XG 310's), while I never used the UTM, the XG is years beyond our old firewall. Not to mention cost, Sophos does a great thing by giving it away to home users. Its a great way to get feedback and help build a more stable platform.
Brian
I have dealt with firewalls for a long time in some form or another(been an System admin for 20+ years).
The functionality Sophos gives you for free is very good. I looked at all the other "free" solutions and kept heading back to Sophos.
I've tried pfsense/opnsense/vyos and they are missing some of the anti-virus/web scanning features without jumping thru major hoops.
Untangle doesn't support ipv6 4in6 tunnels natively, and the good stuff cost $50 a year.
ClearOS/Zentayl are just too complex to keep secure as it tries to do too much.
The only reason i left UTM for XG is the IP limit in UTM and IPv6 addressing chewing up ip's toward the limit.
I really don't want to spin my own.
I'm hoping the 4GB memory issue in XG is worked out soon. Worst case i'd probably fall back to opnsense.
Before this i was using a TomatoUSB firmware variation(shibby) on my netgear WNR3500L v2, but updates have pretty much dried up for that router and it was time to move to something better and something that can run any firewall that can run on X86 hardware. I bought the Zotac Ci323 to run my home firewall of choice. It would be nice to have wireless support in sophos but i know that's asking a bit much.
Maybe sophos could do a low cost per year home use "unlimited" license similar to what Untangle did. Just an idea.
I agree, I would be willing to pay a reasonable yearly fee to have more memory and features, such as access to sandstorm, and Heartbeat on Sophos Free AV. Hopefully Sophos can help figure out soon this kernel limitation.
I too would pay a small nominal fee too, look at ClearOS Home, they have a small fee, 3-5 bucks a month. Very reasonable and to have access to the other features would be great, hopefully Sophos would see the advantage of home users feeding into the larger data analytics which in turn further help their corporate customers.
Guys, while agree with you on Sophos' slow response to the problem, the memory reduction shouldn't affect you that much in reality. I was using XG with 4GB in a vm and it ran fine with about 50ish IPs behind it. Granted I only have a few actual users and most of the other stuff is IoTs and streaming devices, but unless you are a heavy user with a lot of people like a LAN party, 4GB shouldn't impact you severely.
As far as a monthly fee, sophos has been generous to give the product for free. Every few years, someone asks for a home license that would cost 5-10 dollars a month, 100 dollars a year and so on. While I personally don't mind the idea, astaro and now sophos don't want to do home user market. By giving the product away free, they don't have to worry about supporting every home user and then showing them what is a subnet mask. I would rather use a full featured enterprise product with no support than a watered down home version of XG that then I have to pay for to get fewer features that are focused for a home user.
Yeah 4G isn't that big a deal for me in particular, but the licenses explicitly states 6G of memory is allowed and you can have more in the system. The mem=6G addition in my opinion was not fully researched on it's potential impact. If you look up the option in the linux documentation it specifically says it's a address limit. That address limit appears to break the ability to remap around the reserved space. Some people on the internet have said that it's sometimes required to actually set the limit higher that the amount of memory you want to actually get the targeted amount.
If i knew how to test varying kernel options using the kernel and boot loader that sophos uses i would. I may try testing this weekend with an ubuntu 14.04 image (3.13 kernel) and setting mem=6G and see what happens, but it's not an apples to apples comparison.
The fix number quoted by sophos support wasn't even listed in the release notes for MR2, or MR1, which also makes me think it may not have gotten the full attention it deserved.
Makes me even wonder what would happen if mem= was used on their own XG hardware.
I'm very grateful for what they provide. This is simply a bug that needs to be fixed. Their fix to limit to 6G is broken depending on hardware.
In the end they need to find a different solution to limiting memory or check the results of the mem= and increase the limit until the system boots with the wanted amount of usable memory. I'd imagine some program that just steals/allocates the total memory minus 6G might be another idea. There is so little documentation on mem= out there i personally wouldn't use it unless absolutely necessary.
NOTE: I did some experimentation on my firewall hardware using an Ubuntu 16.04 mini ISO which is a newer kernel(4.4)
I booted the kernel with different values of mem=
mem=7G resulted in around 4.2G usable. I didn't write it down sorry.
mem=8G resulted in 5963420 K usablemem=9G resulted in 6995608 K usablemem=10G resulted in 8027796 K usable, which is what I get when booting without mem=
I think this also shows that even with a fairly recent kernel mem= isn't doing what i believe Sophos expected it to do. It simply lops off the ability to map any memory over 6G. If the system memory is mapped outside that range you are out of luck
Here's the memory map from dmesg, i made the values as close as possible. might be rounding errors.
BIOS-e820: [mem 0x0000000000000000-0x000000000009c7ff] usable => RANGE is 0K to 641023(640K) 1M
BIOS-e820: [mem 0x0000000000100000-0x000000001effffff] usable => RANGE is 1048576(1MB) to 520093695(520MB) 519MB
BIOS-e820: [mem 0x0000000020200000-0x000000007b218fff] usable => RANGE is 538968064(539MB) to 2065797119(2048MB) 1509MB
BIOS-e820: [mem 0x000000007b294000-0x000000007b3bbfff] usable => RANGE is 2066300928(2066MB) to 2067513343(2067MB) 1MB
BIOS-e820: [mem 0x000000007bb31000-0x000000007bffffff] usable => RANGE is 2075332608(2075MB) to 2080374783 (2080MB) 5MB
BIOS-e820: [mem 0x0000000100000000-0x000000027fffffff] usable => RANGE is 4294967296(4294MB) to 10737418239(10737MB) 6443MB
In my case the that last memory segment is 6G of my 8G of memory, but it starts at 4GB instead of directly after the previous block at 2G.
When the mem=6G kicks in the last memory segment looses 4G of capacity.
user: [mem 0x0000000100000000-0x000000017fffffff] usable => RANGE is 4294967296(4294MB) to 6442450943(6442MB) 2148MB
AND....
1+519+1509+1+5+6443 = 8478
1+519+1509+1+5+2148 = 4183
My math might be a little off but i think this proves my point without a shadow of a doubt.
If we want to figure out why some are ok, and some are 4G, and others are somewhere in-between i'm pretty certain it's going to map back to where your usable memory maps in.
If you can't get the info from /var/tslog/syslog.log anymore the current RAM ranges can be found using the command below.
SFVH_SO01_SFOS 16.05.2 MR-2# grep RAM /proc/iomem
00001000-0009c7ff : System RAM
00100000-1effffff : System RAM
20200000-7b217fff : System RAM
7b293000-7b3bafff : System RAM
7bb30000-7bffffff : System RAM
100000000-17fffffff : System RAM
I believe this is likely what my bios is doing, and probably many others:
https://en.wikipedia.org/wiki/PCI_hole#Mapping_memory_to_addresses_above_4_GB
NOTE: I did some experimentation on my firewall hardware using an Ubuntu 16.04 mini ISO which is a newer kernel(4.4)
I booted the kernel with different values of mem=
mem=7G resulted in around 4.2G usable. I didn't write it down sorry.
mem=8G resulted in 5963420 K usablemem=9G resulted in 6995608 K usablemem=10G resulted in 8027796 K usable, which is what I get when booting without mem=
I think this also shows that even with a fairly recent kernel mem= isn't doing what i believe Sophos expected it to do. It simply lops off the ability to map any memory over 6G. If the system memory is mapped outside that range you are out of luck
Here's the memory map from dmesg, i made the values as close as possible. might be rounding errors.
BIOS-e820: [mem 0x0000000000000000-0x000000000009c7ff] usable => RANGE is 0K to 641023(640K) 1M
BIOS-e820: [mem 0x0000000000100000-0x000000001effffff] usable => RANGE is 1048576(1MB) to 520093695(520MB) 519MB
BIOS-e820: [mem 0x0000000020200000-0x000000007b218fff] usable => RANGE is 538968064(539MB) to 2065797119(2048MB) 1509MB
BIOS-e820: [mem 0x000000007b294000-0x000000007b3bbfff] usable => RANGE is 2066300928(2066MB) to 2067513343(2067MB) 1MB
BIOS-e820: [mem 0x000000007bb31000-0x000000007bffffff] usable => RANGE is 2075332608(2075MB) to 2080374783 (2080MB) 5MB
BIOS-e820: [mem 0x0000000100000000-0x000000027fffffff] usable => RANGE is 4294967296(4294MB) to 10737418239(10737MB) 6443MB
In my case the that last memory segment is 6G of my 8G of memory, but it starts at 4GB instead of directly after the previous block at 2G.
When the mem=6G kicks in the last memory segment looses 4G of capacity.
user: [mem 0x0000000100000000-0x000000017fffffff] usable => RANGE is 4294967296(4294MB) to 6442450943(6442MB) 2148MB
AND....
1+519+1509+1+5+6443 = 8478
1+519+1509+1+5+2148 = 4183
My math might be a little off but i think this proves my point without a shadow of a doubt.
If we want to figure out why some are ok, and some are 4G, and others are somewhere in-between i'm pretty certain it's going to map back to where your usable memory maps in.
If you can't get the info from /var/tslog/syslog.log anymore the current RAM ranges can be found using the command below.
SFVH_SO01_SFOS 16.05.2 MR-2# grep RAM /proc/iomem
00001000-0009c7ff : System RAM
00100000-1effffff : System RAM
20200000-7b217fff : System RAM
7b293000-7b3bafff : System RAM
7bb30000-7bffffff : System RAM
100000000-17fffffff : System RAM
I believe this is likely what my bios is doing, and probably many others:
https://en.wikipedia.org/wiki/PCI_hole#Mapping_memory_to_addresses_above_4_GB
Kernel Code link for mem=
http://lxr.free-electrons.com/source/arch/x86/kernel/e820.c#L839
