Youtube and ADS - Sometimes they are not blocked

Hi All,

Configure a Web Policy to block "Advertisement" category and enable HTTP & HTTPS scanning. Configure a DROP rule for UDP 443 service and make sure to place it on TOP. YouTube ADS are successfully blocked and during instances of incorrect web categorization, take the steps as suggested by Aditya Patel in his previous post. Refer the screenshots below:

Thanks

Sachin,

I am already using this configuration (apart the UDP 443 blocking rule) which is catched by the default rule at the bottom.

I will check the config and post back again.

Thanks

Thanks Michael,

yours is a great news. I think that you should add multiple and different web filtering engine to XG/UTM, specially for customers that are using Sophos Heartbeat. A dual defense on Web Filtering nowadays is a must and if XG and Endpoint use the same engine is not so safe.

As NGFW, XG should "search" web traffic from more than one DB. Think about to take an agreement with other products. The same applies to Sophos Web Appliance (SWA).

Regards

Hi Michael, as always thanks for the concise thoughtful response. You are right about the URL that I mentioned in my post and I am not saying that in certain cases XG won't classify something better. I also realize that change is hard for some of us folks[:$]

Michael Dunn said:

For those of you with a long memory.  Do you recall that in UTM the original Astaro it used a categorization service known as CFFS.  In 9.2 we switched it to SXL.  It used the same McAfee database but had several improvements, including the addition of Sophos Labs data source for malicious urls.  We are doing the same thing for XG.  Currently XG uses a service called WINGc.  In v17 we will be switching this to SXL.  This will give us the better data source for malicious URLs.  The evolution of XG is following the same evolution as UTM

Thanks again for taking the time and raising the issue with the correct team.

The main change from WINGc to SXL is we are changing the cloud servers and the method of communicating with them.  The data backing them is 99% the same.
The difference is that WINGc (from Cyberroam days) does not contain Sophos Labs security data.
Rather than adding the Sophos Labs data to WINGc, we have added the Cyberoam-now-Sophos categorization data to SXL, and are moving XG to SXL which now has both.
Sophos Labs data is based on emerging threats and is managed more like antivirus data.  Security data is often shared between companies because security is more important than proprietary.
It is a small drop in the total categorization data, yet it is an important one.

Endpoints are complicated because where they get categorization data can change depending on the product they are associated with.  There are now also several different endpoint products.

I'm not in the newsgroups daily, my involvement ebbs and flows with my other workload.  I've got a fine line to walk about what I can/should talk about and not.  I am also sometimes limited in what I can actually do about issues, Support and Partner channels sometimes being the better options.  That being said, I think teasers and insights like this are good.  I'm running SXL on my test v17 XG box right now.  :)

Thanks again Michael for sharing your knowledges and news. In this community, we should see more people like you from each department (Email, WAF, Network, etc...) so you can read forums, get in contact with people here and take notes of new ideas/bugs.

We are all here to share our point of view and to improve Sophos products. Take our criticisms are constructive and not disruptive!

Thank again! We are looking forward to testing v17!

Regards

After almost a year I would inform Sophos and suggest to them to do something with ADS blocking. Web Filtering is not blocking ADS, simple! Why Top management have to pay for something that simply does not work? This is the question I receiving with XG.

Dear Sophos's Guys, you have to improve it. We pay for Web Filtering and I had enough to receive complaints.

I guess I am not alone, here....

Hi Luk

As you might know, I have (when I´m bored or have too much time to waste) as kind of a "hobby" blocking away www advertisers and trackers - I also wrote in the past a "guide" to give people a base to start blocking www ballast away ==> Blocking Ads in UTM

However, besides of the differences in the used URL databases behind UTM and XG, both products are not designed to operate as adblockers as main function. There are some nice features built in to start to make websurfing faster and cleaner...

From my personal experience the XG URL database became quite good over the past 1-2 years. Many of the advertisers URLs or domains gets blocked out of the box using the "Advertisements" category. Maybe if I get some spare time, I might start once with a "tweaking guide" for the XG firewall or a "how to block advertisers, trackers and other WWW ballast" guide...

However, as base to start you might get best results with following rough steps:

- Use HTTPS scanning/MITM in the webproxy whenever possible (as this give more granular control for filtering content)

- Blocking, or at least set to "Warn" for the "Advertisements" Category already blocks lot of the unnecessary ballast away

- I also use a "URL Group" named "Additional_Advertisers_Trackers" containing some additional domains delivering ads or tracking users, which otherwise might be categorized as Content Delivery or General Business in the URL DB, and block them away too. Actually for my surf behaviour I added following domains, which might be a start for your own domain collection:

quantserve.com
spiceworks.com
outbrain.com
cxense.com
ensighten.com
chartbeat.com
visualwebsiteoptimizer.com
gigya.com, adtech.de
clicktale.net
krxd.net
mpstat.us
usabilia.com
supersonicads-a.akamaihd.net
unityads.unity3d.com
supersonicads.com
adcolony.com
external-frt3-1.xx.fbcdn.net
pixel.facebook.com
l.facebook.com
google-analytics.com

However keep in mind, some of those domains also might deliver some legitimate content which also gets blocked away, so use that "mini guide" at your own risk ;o)

Filtering that way the websurfing experience is already quite clean and fluid. I anyway still use "real" adblockers in browsers as uBlock in Firefox to further clean up websites, as adblockers got designed and optimized to get one single and specific task done right.

/Sascha

Hi Sascha, Luk,

I use the XG ad blocking feature and seem to block most ads, though not all and I have to add some exceptions because as Sascha advises some site use ad sites to deliver genuine product.

This was yesterday's little collection. I do note that firefox and safari block different ads after they get through the XG.

Ian

Thanks Sascha for your response but for the web filtering, I have enough...me and of course my customers. XG is deployed using HTTPS scan since v16 but ADS and several URL are not correctly categorized. I encourage my customers to submit false positive using Sophos website but this engine does not its job. I understand that you moved to your own engine, but remember that customers are the more important part here and if they are not happy, they move to something else.

Using custom url blocking is a workaround but you have to sit with the XG console, check logs, reports and submit so many false positive and fill the url group? We have time for that but not all day x 365 days.

Check rfcat_vk feebacks.

Sorry guys but XG web filter at the moment is unusable for its catching quality rate.

On UTM 9 we can block additional ads via application control. Why there isn't an similar option (app classification for web-ads and trackers) in the XG firewall?

I would like to block the Ads in the youtube videos, but from prior discussions on this subject, youtube figured out a way of bypassing ad blocking by incorporating them in the videos. So to block a youtube AD you will end up blocking youtube videos, maybe not a bad idea.

Ian

I would like to block the Ads in the youtube videos, but from prior discussions on this subject, youtube figured out a way of bypassing ad blocking by incorporating them in the videos. So to block a youtube AD you will end up blocking youtube videos, maybe not a bad idea.

Ian