Youtube and ADS - Sometimes they are not blocked

Hi Luk,

Just to confirm are you using Decrypt and Scan HTTPS?

Emile

Thanks Emile. I was not using HTTPS scanning on UTM and even on XG. I reinstalled UTM on a VM and youtube ads never appear while using XG, sometimes they come out.

Really sad!

I noticed similar behavior with youtube and I don't use decrypt and scan either. I didn't research it too much but I have noticed that there is some categorization discrepancies between UTM9 and XG. Here is one example...

While XG offers many categories, some items are categorized wrong and there are many items are under "Dynamic DNS & ISP Sites" category that don't make any sense.

I don't know why sophos doesn't take what has already worked in UTM9 and use it for XG. They keep on reinventing stuff... look at the MTA, they brag about it as one of the great new features in XG that brings it closer to UTM9. Are they kidding me? UTM9 has a full function MTA server while XG has a very basic MTA that is nowhere as capable as UTM9. Why don't they use Exim from UTM9 that has worked for years? I give up[:D]

Regards
Bill

Hi Luk,

For the past half an hour I've just been playing with the XG and have found some interesting things:

• If I set a deny all Web policy on the XG for my firewall rule I'm going out on Youtube is allowed on Google Chrome but blocked on IE
• To block Youtube on Google Chrome I had to physically enable any application filtering policy which is crazy. It was very odd in Chrome, Youtube wasn't even coming up in the logs when I was browsing, wizardry is afoot when browsing Youtube in Chrome
• Advertisements were not being blocked no matter what because after examination of the advertisements in packet streams and via the web filtering the ads are actually youtube.com site paths like /pageads, /annotations_invideo, /api/stats/ads which means the way the XG does categorisation means it cannot examine the site path. I'm very curious as to how the UTM did this now
• I create an Application Filtering policy that denies everything by default and I only allow Youtube Website, Youtube Search and Youtube Streaming i get some interesting results. The ads are blocked ish but only the content of the ad, I still get the pop up but only random ones are blocked and some are getting through. Methinks Youtube has some magic wizardry going on here to bypass scanning like this

Right now I'm try to find some way of entering paths into the XG as areas for blocking but I'm coming up at dead ends. What I need is the ability to block websites based on Regular Expressions!

Hi BillyBob,

What features are missing from the XGs MTA that are in the UTM?

I find the MTA server in the XG to be far more advanced and granular than the systems employed in the UTM if not on the same level of.

Emile

Billybob said:

 

I don't know why sophos doesn't take what has already worked in UTM9 and use it for XG. They keep on reinventing stuff... look at the MTA, they brag about it as one of the great new features in XG that brings it closer to UTM9. Are they kidding me? UTM9 has a full function MTA server while XG has a very basic MTA that is nowhere as capable as UTM9. Why don't they use Exim from UTM9 that has worked for years? I give up[:D]

Regards
Bill

 

[:#]

Bob, we can add your question to "long list" of unanswered questions. Even the crying face emoticon disappeared on this community. Just crying....

Saschin, AlanT are you there to give us more info about those big differences between XG and UTM? I never had any issue on Youtube website and Ads....

Still a lot of work in Sophos Development House!

Emile,

thank you for sharing your test. As you saw, to block something we need to test and retest and nothing in XG is straightforward. For my point of view, if I block ADS, they have to be blocked on every browser and every website. For me Youtube is first a website and then an app.

I am spending time on XG but many things are just crazy. Web proxy on UTM was much better and it works well.

For the MTA part, BillyBob is right. This MTA is still basic one. I understand that this is the first version of XG that integrates MTA but some advanced features are missing such as: SPF, BATV and DKIM; no e-mail address based whitelist/blacklist in SMTP proxy and others....

UTM is much better even managing Profiles and how it manage inbound and outboud scanning. On XG I see to much rules (any -- any --any) which are not so secure.

It still a mess and complicated understand the XG philosophy.

I do not hate XG but I do not have good reason to love it (until now).

Hi Emile, first congrats on the mod title. I know your knowledge and desire to make XG into a great product will be greatly appreciated by sophos and the users of this community. As far as MTA, I wrote about it during beta but didn't hear back from sophos as usual https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-feature-requests/78869/additional-mta-features-needed-in-the-gui 

Honestly, my lab has a dynamic IP address and I can't really test the MTA like I can with UTM due to smarthost capabilities. Before beta5 I believe, they also had other problems like RNDS with outgoing mail https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-issues-bugs/78862/rdns-settings-applied-to-outbound-relaying-hosts 

While XG maybe fine for some deployments, I certainly wouldn't use it as my edge SMTP server. Just the lack of logging is enough to discourage anyone. By the way I thought they were going to improve logging before the GA https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-feedback/78150/ui-and-tabs-it-is-still-complicated-to-find-and-understand-their-relations/304271#304271 

 I didn't notice the cry emoticon. Everything is so good now, we don't need it [;)]

Regards

Bill

Hi Billy,

Cheers, hopefully I can keep up providing knowledge and assistance!

Wow, I really must have glossed over the config with glazed dead eyes and thank you for highlighting the missing features, I am taken aback as to how I did not notice it. I will have to focus some more of my efforts to testing Mail and MTA, I've been 99% concerned with features like web, IPS, App protection, Routing and various other similar low level areas.

I came into Sophos products around the time Copernicus was in Alpha and I learnt Copernicus at the same time as the UTM v9.3 so you could say "I grew up with both worlds". The logging is a massive gripe of mine as well and I also remember the logging resolution before GA as well so I'm holding out for a hero.

On the topic of Ads, Youtube have done some very clever stuff that appears to be bypassing normal detection methods but will spin up the same tests for UTM and see if the same result occurs :)

Emile

Hi Billy,

Cheers, hopefully I can keep up providing knowledge and assistance!

Wow, I really must have glossed over the config with glazed dead eyes and thank you for highlighting the missing features, I am taken aback as to how I did not notice it. I will have to focus some more of my efforts to testing Mail and MTA, I've been 99% concerned with features like web, IPS, App protection, Routing and various other similar low level areas.

I came into Sophos products around the time Copernicus was in Alpha and I learnt Copernicus at the same time as the UTM v9.3 so you could say "I grew up with both worlds". The logging is a massive gripe of mine as well and I also remember the logging resolution before GA as well so I'm holding out for a hero.

On the topic of Ads, Youtube have done some very clever stuff that appears to be bypassing normal detection methods but will spin up the same tests for UTM and see if the same result occurs :)

Emile