[9.200][BUG] HTTPS - Exceptions issue

Looks like something broke....

Exceptions not read when https URL Scan only is turned on for https urls.

IF you have a web protection>webfiltereing options> exception to skip all checks including SSL, URL, etc. and you have HTTPS "Scan URL only" selected in the web protection>>filter profiles, any https scan will not be seen by the exception and return a block.

Example listed below:

Apple Update [Allows Apple Update without content scanning side effects.]
Skipping: Authentication / Caching / Antivirus / Extension blocking / MIME type blocking / URL Filter / Content Removal / Certificate Trust Check / Certificate Date Check / SSL scanning
Matching these URLs: ^https?://([A-Za-z0-9.-]*\.)?apple\.com\.?/

2014:03:26-11:49:47 sbu001 httpproxy[5454]: id="0060" severity="info" sys="SecureWeb" sub="http" name="web request blocked, forbidden category detected" action="block" method="CONNECT" srcip="192.168.1.35" dstip="" user="" statuscode="403" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="0" request="0x27d6f760" url="https://play.itunes.apple.com" exceptions="" error="" authtime="0" dnstime="0" cattime="76" avscantime="0" fullreqtime="15022" device="0" auth="0" reason="category" category="129" reputation="neutral" categoryname="Media Downloads"
2014:03:26-11:49:48 sbu001 httpproxy[5454]: id="0060" severity="info" sys="SecureWeb" sub="http" name="web request blocked, forbidden category detected" action="block" method="CONNECT" srcip="192.168.1.35" dstip="" user="" statuscode="403" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="0" request="0x27d6fdc0" url="https://play.itunes.apple.com" exceptions="" error="" authtime="0" dnstime="0" cattime="77" avscantime="0" fullreqtime="13769" device="0" auth="0" reason="category" category="129" reputation="neutral" categoryname="Media Downloads"
2014:03:26-11:49:53 sbu001 httpproxy[5454]: id="0060" severity="info" sys="SecureWeb" sub="http" name="web request blocked, forbidden category detected" action="block" method="CONNECT" srcip="192.168.1.35" dstip="" user="" statuscode="403" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="0" request="0x27e34ee0" url="https://play.itunes.apple.com" exceptions="" error="" authtime="0" dnstime="0" cattime="99" avscantime="0" fullreqtime="14895" device="0" auth="0" reason="category" category="129" reputation="neutral" categoryname="Media Downloads"
2014:03:26-11:50:23 sbu001 httpproxy[5454]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="192.168.1.35" dstip="23.3.87.120" user="" statuscode="200" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="686410" request="0x27cbc880" url="https://iadsdk.apple.com" exceptions="" error="" authtime="0" dnstime="24315" cattime="52" avscantime="0" fullreqtime="91944618" device="0" auth="0" category="105,175" reputation="trusted" categoryname="Business,Software/Hardware" application=""
2014:03:26-12:06:29 sbu001 httpproxy[5454]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="192.168.1.32" dstip="96.17.202.35" user="" statuscode="200" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="16529" request="0x27e34cc0" url="gspa21.ls.apple.com/.../prod-resources-lodpi-20" exceptions="av,auth,content,url,ssl,certcheck,certdate,mime,cache,fileextension" error="" authtime="0" dnstime="168495" cattime="0" avscantime="0" fullreqtime="252779" device="0" auth="0"
2014:03:26-12:08:34 sbu001 httpproxy[5454]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="192.168.1.35" dstip="174.76.226.82" user="" statuscode="200" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="2" request="0x27cbd760" url="http://gsp1.apple.com/pep/gcc" exceptions="av,auth,content,url,ssl,certcheck,certdate,mime,cache,fileextension" error="" authtime="0" dnstime="135972" cattime="0" avscantime="0" fullreqtime="178480" device="0" auth="0"
2014:03:26-12:24:32 sbu001 httpproxy[5454]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="192.168.1.32" dstip="17.134.126.132" user="" statuscode="200" cached="0" profile="REF_HttProContaInterNetwo (Office Staff )" filteraction="REF_DefaultHTTPCFFBlockAction (Everyone - Default Group)" size="30775" request="0x27d34440" url="https://gs-loc.apple.com" exceptions="" error="" authtime="0" dnstime="21981" cattime="53" avscantime="0" fullreqtime="672146" device="0" auth="0" category="105,175" reputation="trusted" categoryname="Business,Software/Hardware" application=""

Filter profile is full transparent mode, no authentication, https "URL filtering only" checked

0 Michael Dunn over 11 years ago

It looks like a bug in the new feature - not something that was broken from before.

If you are doing "URL Filtering Only" and using Standard Mode then the URL (domain name only) contains a trailing slash.

If you are using Transparent Mode, however, it does not contain a trailing slash.

Because the exception needs a slash it does not match.

This is something to do with how we interpret the domain name or URL in SNI inspection during transparent mode - which is all new.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Astaro Beta Bot over 11 years ago

Thanks for reporting. We are now tracking this as Mantis ID #31245
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Astaro Beta Bot over 11 years ago

The Mantis ID #31245 is now being worked on. We are planning to release a fix for this issue in Version 9.202.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 BartWilsonTwotrees over 11 years ago

Thanks for all your work.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Astaro Beta Bot over 11 years ago

The Mantis ID #31245 is now fixed.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Knome over 11 years ago

CHANGE FROM
^https?://[A-Za-z0-9.-]*\.netflix\.com/
^https?://[A-Za-z0-9.-]*\.hulu\.com/
^https?://[A-Za-z0-9.-]*\.akamai\.net/

            TO
^https?://([A-Za-z0-9.-]*\.)?netflix.com
^https?://([A-Za-z0-9.-]*\.)?hulu.com
^https?://([A-Za-z0-9.-]*\.)?akamai.net

               CHANGE FROM
^https?://([A-Za-z0-9.-]*\.)?itunes\.apple\.com
^https?://([A-Za-z0-9.-]*\.)?mozilla\.net
^https?://[A-Za-z0-9\.-]*windowsupdate\.com/

                 CHANGE TO
^https?://([A-Za-z0-9.-]*\.)?apple.com
^https?://([A-Za-z0-9.-]*\.)?mozilla.net
^https?://([A-Za-z0-9.-]*\.)?windowsupdate.com

it work's great i like it the way its done once
i change every thing to the CHANGE TO example above
site never work better you can access main sites
and all sub sites.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Michael Dunn over 11 years ago

Yes, Knome that would be a workaround until 9.202 is released.

I would note three things:

1) Your examples are mostly exceptions you have written yourself.  The UTM does not ship with exceptions for netflix, hulu, and certainly not akamai (dangerous as it could be anyone).

2) When you change \. to . you are changing a period into a wildcard.  Therefore you are now going to match netflixzcom.

3) By removing the tailing slash you are not terminating the domain name.  That means that you will match netflix.com.some_evil_site.com and your exception will apply.  This is not ideal protection.  It is, however, the workaround you need for this bug.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Knome over 11 years ago

Hi Dunn

yes these are examples are the ones that i have written because was having trouble get to them and a lot more sites.
(certainly not akamai (dangerous as it could be anyone) understood.

Do you have any examples for the work around that might will not match netflix.com.some_evil_site.com and would be ideal protection.

thanks
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Michael Dunn over 11 years ago

Unfortunately I don't think there is a workaround for this bug except that would allow for that type of thing.

The only thing I would do is say this this only affects Transparent Mode when HTTPS is set to URL Filtering Only. There is no need to change anything if you are not using this. After 9.202 is released the exceptions should be modified to include the trailing slash again.

And if you can live with the fact that the exceptions are not applied (eg your customers are not complaining) then don't change anything, just wait for the fix.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Knome over 11 years ago

ok will wait for fix
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel