I really want to like this product, but...

I'll combine your first comment with your #5 first - if you had bought this for your workplace, you'd likely be entitled to support. Support wouldn't be non-existent, and they'd work with you through these issues. I'm working with their support engineers now on a few bugs, and they're excellent. How they should be supporting other folks - like forum folks - that aren't paying XG customers yet - that's a different discussion. I think their involvement here could (should?) be better... especially since they seem to want people to use the free home version... and I think it will get better... Support just got handed a new product, too, so they are working to understand this new XG product as well. Give it a little time.

Now - 1) webfilter issues. Just about any webfilter with HTTPS filtering is going to break something like netflix and amazon fire. I enabled Watchguard's version of https filtering at my house - had very similar issues. I came from and am certified in the watchguard product, so I had a spare of their smaller boxes lying around to test. Anyway - anything that tried to validate SSL certificates - and for which installing new certificates was nearly impossible - generally had major issues. Amazon Kindle e-readers would freak as well. This is a known issue with just about any deep packet inspection https filter, and the only way resolve that is to install the ca certificate from the firewall onto the end devices - or except the traffic from the https filter. The XG device actually makes it fairly easy to except certain traffic from HTTPS inspection. Entire categories can be exempt via the policy, and when ssl certificates for those sites are detected, the traffic is not intercepted and monkeyed with, or individual IP addresses can be exempt as well. I've found it works acceptably well. I have had to add in a few *special* exemptions - logmein rescue, for example, certain sites using websockets in an odd way, that sort of thing - but overall, it is working like an absolute champ.

2) can the gateway be pinged? The xg relies on pings - not whether traffic is passing through the interface - to determine the status of the gateway, I believe. I think I remember that from the XG certification course.... Maybe that was only multi-wan was setup...

3) What's the load average look like on the performance screen?

4) Do a packet capture on the smtp traffic to see what the SMTP conversation looks like. Could be any NUMBER of things - including user error (hey, it happens... even on a stupid smtp connection. )

I would like to thank you for taking the time try and explain things to us. I do understand this is a new product in fact its very much v1 as has been said many times in the beta program when I was a part of it that said communication from Sophos even in the beta program was none existent and when a company is asking for are help in testing and asking for our feedback and it falls a dead ears with no response that's never a good thing.

As for Netflix not working on mobile devices this is something going back to UTM9.x at least in UTM9 we could fix it with some Regex entries and move on with our lives without completely disabling web filter and HTTP scanning, Its not just when HTTPS scanning is enabled its when plane HTTP scanning us used. No matter what we do with different combinations if you have ANY of the following 3 things active it breaks Netflix on mobile devices

web filter
HTTP Scanning
HTTPS Scanning

The difference is in the reports you get. Your way works - and is the quickest way. My way lets you easily identify the ip addresses with user names/descriptions if/when you run traffic reports. Either works.

And before we throw Sophos under the bus for their HTTP/S filtering - lets ask ourselves - is any vendor ACTUALLY PRODUCING a true HTTP and HTTPS filter that works *well* with highly advanced streaming services like netflix? The answer is likely no. Netflix in particular is a very odd bird due to the way it streams using direct IP addresses instead of host names that can easily have regex or string comparisons done against them.

I may open a support ticket on this and see what support says. I may also put my challenge on hold until the new version is released in January/February... see what they fix/change. I don't expect them to FIX it outright, but they very may well change something that either breaks whatever I come up with *OR* fix something else (like regex) that allows me to make things actually work.

Thank you for the detailed explanation. That would be great if you could enter a support ticket. I think the more we hound Sophos to get this fixed once and for all the better. I look forward to any updates you may have.

Thanks again for taking the time...

Hi ChavousCamp,
Unrelated to Netflix, Do you know to fix I am having trouble with Windows downloading new Insider builds when the device is behind XG, The only way I can download the new builds is to disable HTTP and HTTPS scanning. When I go into the logs XG does not show anything blocked. What do I have to do to fix this so I don't have to DISABLE HTTP scanning?

Thanks

Until they allow multiple regex expressions per single rule this will be a continuing problem. It was brought up multiple times during beta testing and I have already addressed my feelings on that subject so I won't hijack this thread.

Regarding no blocks showing in logging, refer to this thread https://community.sophos.com/products/xg-firewall/f/46/t/15732 by  

billybob, Thanks for the info. I agree with you 100% but now the million dollar question, Will they fix it? It was said so many times on the beta forum that this product was not ready for prime time but ready or not they role it out anyway. I guess that's the price you pay for early adoption and I am more then willing to hang in there and keep using it as long as they fix things in a timely manor and don't make us wait years. I can only hope that the next update due out in Jan-Feb addresses some of this stuff.

how is this going? I have heard only one regex is supported, so I expect it will be necessary to use the 'or' symbol so mush the 'dreashna' items into a single regex....

"Will they fix it" -

That depends on what you call "fix."

*FIX* regex by either adding it to policies or making it actually *work* on the web filter screen so someone *can* allow netflix through? Probably. If not in Jan/Feb then at some point soon. Please lets hope, and for reasons other than netflix.

-OR- do you mean

*FIX* the XG so that it is easier to allow netflix to get through - doubtful.

I've seen folks in this thread and elsewhere lambaste Sophos - the XG and UTM product line - that it is too hard to allow certain streaming services and that it breaks x or y when https filtering is enabled. I really don't think folks realize THAT'S BY DESIGN. Lest we forget, this is *NOT* a product explicitly designed for home use. I love the fact that they are giving it away for technophiles and the like to play with at home, but lest we forget - it is primarily designed as a business product. Compare it to every other enterprise-grade web filter... Cisco, Websense, Edgewave's iPrism - you'll be hard-pressed to find one that will *EASILY* allow the netflix video streaming through without exempting the entire device from policy. Lets be clear: this IS NOT the fault of Sophos or the other vendors. It is primarily that 1) nextflix is NOT a business priority, 2) netflix does NOT make it easy to identify their hosts and thus allow them and 3) mobile devices are NOT very friendly to HTTPS filtering due to the certificate issues discussed previously.

Again - this is NOT a sophos problem. The reason Sophos is getting hammered on it is that they actually encourage home users to use their product. They probably should publish some caveats with that - and some better documentation and better communication, because what they've done is created a bunch of frustrated, upset users. And that is bad and sad.

I keep repeating this bit about HTTPS filtering and other vendors because I think it is an important security issue.... bypassing this "by design" security by allowing netflix through can have severe consequences. If you take a look at the drashna.net rules for a bit, you may be able to see the security issue... and why I believe his post is called an exercise in futility.

ANY firewall - XG, SG, etc - that implements those rules effectively blesses every IP address on the entire net, and only filters based on the "filename" portion of the URL. It would be relatively trivial for a piece of malware to mimic the netflix - and similar video sites - url structures in order to bypass filtering, because there is literally no host/domain name checking on it.

Honestly, based on the URLs I saw going across the wire when I tested - I'm not even sure the drashna rules will work any more - as the URLs I saw in my testing would NOT match those filters... we'll just have to see once Sophos fixes the regex filter.

I will tell you what an unnamed agent at Sophos support told me:
He determined - and whitelisted - a hundred or so IP addresses associated with netflix, and that's how HE got it to work at his house.

sure, netflix may not be a service that a business needs to allow through, but consider it a test case for how well the product supports a tech person trying to accomplish a goal with the product - is there sufficient information in the log to assist with the diagnosis? are the mechanisms provided sufficient to achieve a desired result? Do they behave as advertised, are they predictable and flexible enough for a particular task? many of us have dual lives and to earn our 'yes' vote at work, a home product may do the trick. Or the reverse. I can easily see setting up some cloud service (or hybrid cloud) as causing similar difficulties as netflix. Opportunity is the word, and meeting it by burying one's head in the sand and yelling 'free home product' does not impress me in the least.

Great points by everyone. Now I am excited for the next update.

Thanks to everyone for taking the time to reply...

Great points by everyone. Now I am excited for the next update.

Thanks to everyone for taking the time to reply...