Bad URL Categorization

This thread is covering two different issues.

1) The initial main issue seems to be a case where some people were getting no category when in fact the sites had been correctly categorized.  I was not involved in the follow-up but I am guessing this was a configuration or connectivity issue.

2) Some people are using this thread to generally report incorrect categorization.

Now as far as I know the initial problem 1) has been resolved.

In the case of 2) then I suggest that if you want to discuss categorization feel free to do it here (or start a new thread).  But if you are just reporting then here is the better place to do it:  secure2.sophos.com/.../contact-support.aspx  Click "Submit a Sample".

You may note that Sophos has three web products - the SWA, the SG UTM, and the XG UTM.  All three actually use different sets of categories that come from different sources.  The main reason for this is history and the pain of having administrators change their policies if we change the list of categories.  One of the things we do is check top hits and do a three-way comparison.  If two of the three categorizations say one thing and the third says another, we mark it as needing re-analysis.  However if all three come up with different answers then we don't - which is actually the case at least two of the sites in ITSUITSU's post.  In order to prioirize work, we use the websites that are more commonly hit by our customers first, and we concentrate more on the commonly blocked sites such as malware.

There are the following challenges
A) We have a database of millions of categorized sites.  Some may be wrong.  Comparison with others and customer reports help identify.
B) There are 571 new websites created every MINUTE.  We have to concentrate on the sites our customers visit most often.
C) Automated analysis tools have to be used to handle the workload, with fallback to humans.  They may be good at analyzing some things (this is a forum) and not others (the forum is discussing white power, which is a higher priority categorization).

Needless to say, we're doing the best we can.

I had brought up this thread as an exapmple of spam on the sophos board when talking to robert lanson earlier this year. I think some people are posting genuine concerns but others with only ONE POST are using it to advertise pornography. A MOD should lock the thread.

Bill, 

send me a PM, thanks.

I would like to understand more about this thread.

Hi Sophos

Really, Sophos does not understain the real issue.

The same URL's are categorized correctly by Sophos UTM9, Sophos Web Appliance, Sophos Endpoint Web and Sophos Home.

For example, in Sophos XG Firewall the same URL's detected as Porn/Nudity o Sexual in the another Sophos products, are detected incorrectly.

So, the issue is that Sophos XG Firewall have a poor "Categorization URL system".

The solution:

Change the XG Firewall Categorization system for the "Sophos UTM URL system".

Cyberoam have the same issue = Sophos XG Firewall poor "Categorization URL system".

Regards

Linck Tello Flores

www.innovare.pe

I hope that in Sophos they will change the web engine. The one used on UTM works like a charm.

Really few false positive/negative on UTM9.

Any reply from Aditya Patel or sachingurung ?

Hi Luk,

I think that will be a debate if we discuss that as Web Categorization on UTM comes from McAfee engine alongside, in XG the categorization takes place from our private managed Web engine which is incorporated from our legacy Cyberoam iNG series. With WINGc categorization in the XG we can quickly change the bad categorization of any website with the help of our internal teams in a day. 

Thanks

Hi Luk,

I think that will be a debate if we discuss that as Web Categorization on UTM comes from McAfee engine alongside, in XG the categorization takes place from our private managed Web engine which is incorporated from our legacy Cyberoam iNG series. With WINGc categorization in the XG we can quickly change the bad categorization of any website with the help of our internal teams in a day. 

Thanks

sachingurung 

Nothing against this decision but make sure to improve it alot. UTM uses multiple web filters (at least it was like that on old Astaro version).

Make sure to improve the url filtering database and follow the customers request. Web Filtering is one of the components where users can experience issues and complain with their System Admin.

I would suggest you to create a proper web page or even better a link within the XG where admins can submit the request directly to your team in order to analyze the request and make the proper changes.

We are looking forward to hearing from you.

I support the lferrara last comment.
Really, follow the customers request.
Sophos must be check immediately the Web Categorization Engine in XG Firewall, simply is BAD. No wait more time!!

If you can't manage this case, you can a buy a best URL list in internet and integrate to XG Firewall.

Or, use in XG the same UTM9 Web Engine, is simply as this, uses the same UTM9 engine en XG not the Cyberoam Engine (this engine is bad).

This case is open from the v15 and after months the issue is the same.

I think that you must include in this at XG Product Manager to take a inmediatelly action.

Regars

Linck Tello Flores

Hi Luk, 

Let us check on this matter and get back to you .