Youtube and ADS - Sometimes they are not blocked

Hi Luk,

Just to confirm are you using Decrypt and Scan HTTPS?

Emile

Thanks Emile. I was not using HTTPS scanning on UTM and even on XG. I reinstalled UTM on a VM and youtube ads never appear while using XG, sometimes they come out.

Really sad!

Hi Luk,

For the past half an hour I've just been playing with the XG and have found some interesting things:

• If I set a deny all Web policy on the XG for my firewall rule I'm going out on Youtube is allowed on Google Chrome but blocked on IE
• To block Youtube on Google Chrome I had to physically enable any application filtering policy which is crazy. It was very odd in Chrome, Youtube wasn't even coming up in the logs when I was browsing, wizardry is afoot when browsing Youtube in Chrome
• Advertisements were not being blocked no matter what because after examination of the advertisements in packet streams and via the web filtering the ads are actually youtube.com site paths like /pageads, /annotations_invideo, /api/stats/ads which means the way the XG does categorisation means it cannot examine the site path. I'm very curious as to how the UTM did this now
• I create an Application Filtering policy that denies everything by default and I only allow Youtube Website, Youtube Search and Youtube Streaming i get some interesting results. The ads are blocked ish but only the content of the ad, I still get the pop up but only random ones are blocked and some are getting through. Methinks Youtube has some magic wizardry going on here to bypass scanning like this

Right now I'm try to find some way of entering paths into the XG as areas for blocking but I'm coming up at dead ends. What I need is the ability to block websites based on Regular Expressions!

Hi BillyBob,

What features are missing from the XGs MTA that are in the UTM?

I find the MTA server in the XG to be far more advanced and granular than the systems employed in the UTM if not on the same level of.

Emile

Billybob said:

 

I don't know why sophos doesn't take what has already worked in UTM9 and use it for XG. They keep on reinventing stuff... look at the MTA, they brag about it as one of the great new features in XG that brings it closer to UTM9. Are they kidding me? UTM9 has a full function MTA server while XG has a very basic MTA that is nowhere as capable as UTM9. Why don't they use Exim from UTM9 that has worked for years? I give up[:D]

Regards
Bill

 

[:#]

Bob, we can add your question to "long list" of unanswered questions. Even the crying face emoticon disappeared on this community. Just crying....

Saschin, AlanT are you there to give us more info about those big differences between XG and UTM? I never had any issue on Youtube website and Ads....

Still a lot of work in Sophos Development House!

Emile,

thank you for sharing your test. As you saw, to block something we need to test and retest and nothing in XG is straightforward. For my point of view, if I block ADS, they have to be blocked on every browser and every website. For me Youtube is first a website and then an app.

I am spending time on XG but many things are just crazy. Web proxy on UTM was much better and it works well.

For the MTA part, BillyBob is right. This MTA is still basic one. I understand that this is the first version of XG that integrates MTA but some advanced features are missing such as: SPF, BATV and DKIM; no e-mail address based whitelist/blacklist in SMTP proxy and others....

UTM is much better even managing Profiles and how it manage inbound and outboud scanning. On XG I see to much rules (any -- any --any) which are not so secure.

It still a mess and complicated understand the XG philosophy.

I do not hate XG but I do not have good reason to love it (until now).

Hi Emile, first congrats on the mod title. I know your knowledge and desire to make XG into a great product will be greatly appreciated by sophos and the users of this community. As far as MTA, I wrote about it during beta but didn't hear back from sophos as usual https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-feature-requests/78869/additional-mta-features-needed-in-the-gui 

Honestly, my lab has a dynamic IP address and I can't really test the MTA like I can with UTM due to smarthost capabilities. Before beta5 I believe, they also had other problems like RNDS with outgoing mail https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-issues-bugs/78862/rdns-settings-applied-to-outbound-relaying-hosts 

While XG maybe fine for some deployments, I certainly wouldn't use it as my edge SMTP server. Just the lack of logging is enough to discourage anyone. By the way I thought they were going to improve logging before the GA https://community.sophos.com/products/xg-firewall/v16beta/f/sfos-v16-beta-feedback/78150/ui-and-tabs-it-is-still-complicated-to-find-and-understand-their-relations/304271#304271 

 I didn't notice the cry emoticon. Everything is so good now, we don't need it [;)]

Regards

Bill

Hi Billy,

Cheers, hopefully I can keep up providing knowledge and assistance!

Wow, I really must have glossed over the config with glazed dead eyes and thank you for highlighting the missing features, I am taken aback as to how I did not notice it. I will have to focus some more of my efforts to testing Mail and MTA, I've been 99% concerned with features like web, IPS, App protection, Routing and various other similar low level areas.

I came into Sophos products around the time Copernicus was in Alpha and I learnt Copernicus at the same time as the UTM v9.3 so you could say "I grew up with both worlds". The logging is a massive gripe of mine as well and I also remember the logging resolution before GA as well so I'm holding out for a hero.

On the topic of Ads, Youtube have done some very clever stuff that appears to be bypassing normal detection methods but will spin up the same tests for UTM and see if the same result occurs :)

Emile

BillyBob is right. Many things are missing on this product but the basic is one: "ust the lack of logging is enough to discourage anyone".

The desire to sell and advise XG is not enough for us, for people that come from Astaro and from people that also manage other appliances and see the big differences with this basic product.

The Bill's post is really well done on the Email part. Email Filtering on UTM is very powerful and very popular. I have seen many UTM9 installation where only the Email part was licensed and used.

Hi Luk, I don't know how much free time do you have but you should PM to get in touch with the right person AlanT??? for the moderator title. I for one appreciate your knowledge and in depth analysis of not only sophos but other competing vendors and general trends in technology. You would make a great mod and your enterprise knowledge in areas like active directory integration, IPS and complicated network routing is very hard to find even when you pay money for it. You will be testing XG till v17 anyways, might as well have some fun [:D]

Regards

Bill

Thank you Bob but your answer does not fix the CAA and OTP issue. ;-)

This is not the right place to talk about becoming a moderator or what else. When I have time I do like to share my experience and knowledges with all of you because Security and IT stuff is my passion (like all of us here).

Without passion and background knowledges the path is very short and cannot go further. I am happy that you appreciated my knowledges here... Community is helping each other to understand product and share point of view/knowledges.

Thank you again, Bob.

I will keep testing XG until v17 and see if the product is an Enterprise one or not.

Hi Emile,

I had the same issue in regards to Youtube working on Chrome no matter what was set. One of the fine folks on this forum (Sachin) helped me out; it turns out it is using an experimental transport protocol over UDP that google came up with called QUIC. It specifically goes over UDP port 443. If you drop traffic on that youtube will be forced back over http/s and you can filter it as normal. 

You can try putting in that rule and see if it solves your other issues.

You can see my question here: community.sophos.com/.../chrome-allowing-some-users-to-bypass-web-filter

Cheers,

Devon

Hi All,

I want to give an update on Youtube and ADS.

It's 4 months I am using decrypt and scan and since last month, Youtube ADS are appearing again. ADS category is blocked (indeed on other websites ADS are blocked) but on youtube video, they are not blocked.

Can someone confirm it?

Even in Sophos, make sure the Web engine is working correctly and it is updated correctly. UTM9 is much better on doing web filtering.

An improvement of catching rate is needed!

Thanks

Hi All,

I want to give an update on Youtube and ADS.

It's 4 months I am using decrypt and scan and since last month, Youtube ADS are appearing again. ADS category is blocked (indeed on other websites ADS are blocked) but on youtube video, they are not blocked.

Can someone confirm it?

Even in Sophos, make sure the Web engine is working correctly and it is updated correctly. UTM9 is much better on doing web filtering.

An improvement of catching rate is needed!

Thanks

sachingurung , 

any update for not blocked ADS on youtube video?

Thanks

HI All, 

To clear some Confusion, UTM used McAfee database for Web categorization. For XG we use our own . So to resolve this issue you may need to submit for Website reassessment. You may need to refer the KB article for instructions and should be sorted out in 3-5 Working Days . 

http://www.sophos.com/en-us/support/knowledgebase/119440.aspx

Aditya,

This is not an answer. We know that Utm 9 is using a different engine. We do not like the web filtering catching level on XG at the moment. Ads are displayed even if they are blocked. In order to submit ads for YouTube website I have to keep the log viewer opened and catch the URL when then ads comes out. Not a big deal, Aditya. Make sure to understand our request and to improve catching level. Customers pay for XG license and when web filtering is not working well, chiefs complain instantly.

In addition:

UTM9 one of its strengths point is the Web Filtering. I know many customers that are not happy with other UTM web filtering module and they inserted the UTM in transparent mode between their UTM and the LAN network) in order to get a proper Web Filtering solution.

Developing on your own the web engine can be cost saving but quality at the moment is not as like an enterprise product that is not the market from many years and does it very well. It is like Cyberoam starts to develop AV Engine and it wants to compare with Sophos which is a leader and is on the Endpoint Market from more than 20 years.

Think about it!

I am all for sophos using their own categorization engine. Perhaps we can meet in the middle and give admin a choice to choose between sophos or mcafee engine just like the av engines? I fully agree with Luk that web categorization engine in XG is not as effective as UTM9. 

Good point! I think that maintaining 2 Web Filters engine and also by using McAfee they have to pay the agreement.

At the moment XG Web Filtering has to be improved a lot and you cannot ask to customers to find out which are the URL not categorized correctly. This can happen for few URL but not for many.

Regards

Nothing changed or improved?

Still ADS on youtube and customers complain. [:@]

Do not ask me to open a ticket for that!

Make sure the Web Engine works as expected and at the same catch level as UTM9.

Thanks