This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Decrypt and Scan HTTPS invalidates HTTPS certificates

I suppose I need to better understand Decrypt and Scan HTTPS Malware Scanning.  I noticed that when browse HTTPS site  the cert is replaced by the Sophos Cert.  So, my question is why and how to troubleshoot. If I turn Decrypt off then all is fine.



This thread was automatically locked due to age.
Parents
  • This is the way HTTPS filtering - just about ANY HTTPS filtering - has to work. HTTPS is an encrypted protocol. Normally, a "middle-man" - like a firewall - cannot snoop on the traffic at all. It has no way to decrypt the traffic, as the two endpoints negotiate the encryption algorithm and pass keys back and forth in such a way as to keep someone from just snooping and impersonating one side or the other.

    The only way, then, to decrypt and scan that traffic, is to perform what, in the wild, anyway, would be the equivalent of a "man-in-the-middle" attack. The firewall therefore acts as the "client" and talks to the secure site, and generates a "fake" certificate to talk to the client, thus impersonating the secure site.

    To resolve this: Trust the root certificate of the XG box. Then the https certificates will appear valid in your browser.

    You will have this problem on ANY device - XG, UTM, Watchguard, etc - that performs deep HTTPS inspection.

  • Sorry to be ignorant, how do I trust the XG root cert? Also, thank you for the explanation.
  • Shamelessly copying this from one of my other posts on this forum.....


    Go here: Protection > Web Protection > Web Content Filter, find HTTPS Scanning CA. See which one is listed. (I can't remember which is the default, and we use a custom CA here)

    Then go here: Objects > Identity > Certificate Authority, find the CA listed above and click the download button on the far right.

    Then, install that onto each computer in the trusted root store. See KB article here: www.sophos.com/.../42153.aspx for instructions. Those instructions are for the web appliance, but starting at the end of #1, it becomes generic.

    Note - this is not perfect. There are devices - like the roku and others - that have their certs built in and there ain't crap you can do for it.

    As such, to exempt a specific device from policy, you can either create rules for them based on their IP, or create clientless users here: Objects > Identity > Clientless Users for your other devices and then create one rule exempting the clientless users group from from https filtering. You can also create a separate wireless network on a separate network segment and exempt all of that traffic from scanning - almost like a guest wifi or something. There are options.

    Clientless group instructions:

    1) Create a clientless users group
    Objects > Identity > Groups
    Add.
    Enter a group name.
    Group Type: Clientless
    Quarantine Digest: Disable
    Save.

    2) Create Clientless users for each exempt device.
    Objects > Identity > Clientless Users
    Add (or add range, if they are in a specific range)
    Enter a username - something descriptive for the device (ex: ccamp-iphone)
    IP Address: the internal ip address
    Group: The clientless group you created in step 1
    Name: Some name. Descriptive. "My Iphone"
    Email: fake an email address. Next version won't require this.
    Description: More useless description info. Not required.

    Click the plus sign if you need to add additional devices.

    Click save.

    3) Create security policy
    Security Policies
    Click on your HTTPS filter rule, click the plus sign, click "Above (User/Network Rule)"
    About this rule---
    Name: Allow Clientless to Bypass Filter
    Identity---
    Match Rule based on user identity: On
    User or Groups: Clientless Group created in step 1
    Source---
    Zone: Lan
    Networks: Any
    Services: HTTP,HTTPS, others if you need them, but these suffice for this walkthrough
    Destination---
    Zone: WAN
    Networks: Any
    Malware Scanning---
    Scan FTP OFF
    Scan HTTP OFF
    Decrypt & Scan HTTPS OFF

    Save.

    This bypasses the specific clientless devices you created from the webfilter entirely. This is actually a reasonably good solution - and may be the "best" solution for roku/appletv/chromecast and other fixed devices that do not regularly leave your network.
Reply
  • Shamelessly copying this from one of my other posts on this forum.....


    Go here: Protection > Web Protection > Web Content Filter, find HTTPS Scanning CA. See which one is listed. (I can't remember which is the default, and we use a custom CA here)

    Then go here: Objects > Identity > Certificate Authority, find the CA listed above and click the download button on the far right.

    Then, install that onto each computer in the trusted root store. See KB article here: www.sophos.com/.../42153.aspx for instructions. Those instructions are for the web appliance, but starting at the end of #1, it becomes generic.

    Note - this is not perfect. There are devices - like the roku and others - that have their certs built in and there ain't crap you can do for it.

    As such, to exempt a specific device from policy, you can either create rules for them based on their IP, or create clientless users here: Objects > Identity > Clientless Users for your other devices and then create one rule exempting the clientless users group from from https filtering. You can also create a separate wireless network on a separate network segment and exempt all of that traffic from scanning - almost like a guest wifi or something. There are options.

    Clientless group instructions:

    1) Create a clientless users group
    Objects > Identity > Groups
    Add.
    Enter a group name.
    Group Type: Clientless
    Quarantine Digest: Disable
    Save.

    2) Create Clientless users for each exempt device.
    Objects > Identity > Clientless Users
    Add (or add range, if they are in a specific range)
    Enter a username - something descriptive for the device (ex: ccamp-iphone)
    IP Address: the internal ip address
    Group: The clientless group you created in step 1
    Name: Some name. Descriptive. "My Iphone"
    Email: fake an email address. Next version won't require this.
    Description: More useless description info. Not required.

    Click the plus sign if you need to add additional devices.

    Click save.

    3) Create security policy
    Security Policies
    Click on your HTTPS filter rule, click the plus sign, click "Above (User/Network Rule)"
    About this rule---
    Name: Allow Clientless to Bypass Filter
    Identity---
    Match Rule based on user identity: On
    User or Groups: Clientless Group created in step 1
    Source---
    Zone: Lan
    Networks: Any
    Services: HTTP,HTTPS, others if you need them, but these suffice for this walkthrough
    Destination---
    Zone: WAN
    Networks: Any
    Malware Scanning---
    Scan FTP OFF
    Scan HTTP OFF
    Decrypt & Scan HTTPS OFF

    Save.

    This bypasses the specific clientless devices you created from the webfilter entirely. This is actually a reasonably good solution - and may be the "best" solution for roku/appletv/chromecast and other fixed devices that do not regularly leave your network.
Children
  • You are correct in a limited way I do not have a deep understanding as you provided. All your post did is re-enforce my original statement. If an XG or UTM 9 can do the interception, why can't a bad guy using the same techniques? Isn't that how NSA and others have got access to data in encrypted streams? The only way is to build your own encryption between you and the people who are important to you with no allowed certificates except the one you provide otherwise your trust of the connection is broken.
  • (sigh)

    That is ONE extreme level of security, but of you require that level of security, you need to go ahead and disconnect yourself from the internet and create a private network like the gov't does in a SCIF. You aren't going to find that level here.

    The KEY to the SSL/TLS security is the fact it is "secure enough." Your BROWSER or OPERATING SYSTEM is supposed to alert you when a connection is not *trusted*. Thus, NSA and black hats would NOT be able to just bypass SSL... WITHOUT YOUR KNOWLEDGE. Can they? Yes. Will you be alerted to a potential threat? YES.

    You can probably even set your browser to reject untrusted certs outright. I think there is a policy setting in there somewhere.

  • Regarding ChavousCamp's instructions....

    I just put in an XG115 and enabled the decrypt and scan option and of course you can't go to any https URLs.  So my initial reaction was, what is the option there for if it doesn't *easily* work?

    So I see ChavousCamp's instructions after searching and my reaction, are you kidding me?  I have to go around to all the machines and install certificates?  

    So I take it this option is there for those that want a more robust layer of protection that are willing to do all that extra work.  For me, I paid GOOD money for this device with the intention that it offers some protection AND makes my life as a Network Admin, from the security standpoint, EASIER not HARDER!

  • Good security isn't easy, Jeff.  You could have paid good money for any number of other manufacturers' devices, whose names I shall not mention here, and you would be in the same boat.

    With all due respect, perhaps you should blame yourself for not doing your research on what was REQUIRED to make such decrypt-and-scan functionality work with ANY vendor, or blame yourself for not having the knowledge of how HTTPS works and why this is required.  Don't blame Sophos - or any of the other vendors - for the difficulty level of implementation on your network.  It is the same for all of them.  Sophos actually makes it REALLY easy to set up ON THEIR BOX, and then to MAINTAIN, TROUBLESHOOT, AND REPORT.  That's where the ease of use comes in.  

    Fortunately for security, unfortunately for us trying to scan/filter it, the only way to decrypt and scan https traffic is to, effectively, perform a man-in-the-middle attack on the traffic.  HTTPS, with its certificates, was EXPRESSLY designed to try to prevent us from doing what we are doing.... and that is a *good thing* most of the time.  

    If you are running a domain environment, it is easy - add it to GPO.  If you are not, then yes, some work will be required.  

  • Is it really that easy on a domain?  some browser don't use the windows trusted root store, also mobile devices aren't domain joined

  • Sixteen ,

    Checking https is more needed those days where malware are using https connection to hide themselves.

    I am buying certificate from public ca to prevent the error on clients, specially on mobile.

    Thanks

  • Did you buy a certificate that XG uses to sign its own on the fly generated https certificates?  Any mitm hacker would love to order one too!

    Or just a certificate to open the Sophos webpage without errors?

  • Sorry to all,

    but T9 while I was driving made a big mistake.

    Sorry about that.

    As wrote, for decrypt and scan you have to import the XG certificate inside the Computer Browser. At the moment, XG will show the IP address and not the DNS name. We hope that in Sophos they fix it soon.

    I am also buying certificate for the rest of the services.

    Sorry if I do not add more details but I am writing from mobile.

    I deleted your reply because you used dirty word.

    Regards

  • Let me try a simpler version.

    1) HTTPS uses encryption.  It is designed so that know one can listen into the conversation without you knowing.

    2) If you go to an HTTPS site and it works, then your browser believes that only people who have permission can decrypt the conversation.

         2a) It does this because HTTPS:site.com says it is "signed" by a Certificate Authority which your browser trusts.

    3) If you to to a site and someone is doing a man-in-the-middle attack and decrypting your communications, your browser gives you a warning.

         3a) This is because it is signed by a Certificate Authority you do not trust.

    4) That warning prevents evildoers and governments from decrypting.  But you could ignore the warning and just keep going.

    5) If you decide that having a specific entity listening into the communication is acceptable and you don't want to be warned every time they do, then you install their CA (Certificate Authority).

    6) Now every time a site is signed by the CA doing the man-in-the-middle your browser is happy because the certificates are signed by someone they trust.

     

    Installing the CA can be done manually on each computer, or via an active directory push.  You can also do things like host the CA on your own internal site and give a link to it in block pages and such so that people can download it and install it easier.

    NSA could use the same technique as the XG, but it would still require an action from the computer's admin to install the CA.

  • Eh... short answer, sixteen again, is NO, it is still not "that easy...."

     

    BUT it is easier.

    Here's the thing - and I have been meaning to write about it for a while and just haven't -

     

    ** BUYING A SECURITY APPLIANCE - ANY SECURITY APPLIANCE - IS NOT A QUICK FIX TO ANY PROBLEM. ** 

     

    They are **ALL** designed - even that god-forsaken fish one that squawks on my radio channel all the time about how awesome it is and how it is a be all and end all solution - to be SINGLE PIECE of an overall security policy. 

    Security policy includes - 

    • Source of your devices? Corporate owned or BYOD?
    • Location of your data? "Cloud" or on-prem?
    • How is that data accessed?  
    • Do your BYOD users *just* access the internet or do they access local systems?
    • How do you protect your data outside of just your firewall? Encryption? Backups? Disaster Recovery?
    • What ZONES do you have on your network? Where will traffic for wireless and other devices flow?
    • What APPS do you have on your network that may required unfettered HTTPS traffic?
    • What enterprise management systems do you have in place (roughly lumping Active Directory in that category)?  Do you have an MDM/EMM solution? Do your users bring their own devices exclusively?  Do they use mobile to access data ON YOUR NETWORK or just to browse the web?  

    Etc, etc etc.

    I do ** NOT ** speak for Sophos, but I can say that ** THIS IS EXACTLY WHY ** Sophos only sells their hardware devices through authorized re-sellers.   Security is complicated.  There is A LOT to think about and a lot to know.

    This is also **EXACTLY WHY** Sophos sells multiple layers of products - Endpoint, Encryption, Network Edge, Wireless, Mobile Device Mgt.  That is also *EXACTLY WHY* Sophos has spent so much time and effort INTEGRATING all those components so they work nearly seamlessly together. None of them are designed to be deployed in isolation with any sort of expectation that it is a fix for ANYTHING.

    If you are using an EMM/MDM solution, you may be able to push a CA cert with that system similar to how AD does it.  If you are using enterprise deployment policies for Apple products (YMMV - I know very little about them), you may be able to push a cert onto iOS and OSX devices. Google may have similar for chromebooks.

    Everyone loves BYOD, but BYOD has to include some CONTROL - ie an MDM/EMM solution - that keeps data secure and enforces company policy, which can include things like enterprise CA installation.

    Even that is not perfect, that still will not solve all the issues.  I have apps that *absolutely refuse* to be handled properly under HTTPS decrypt-and-scan.  Anything that validates its own certificate before connecting - like some of the Amazon Web Services SDKs - will likely break, because they are not just looking for a valid cert, but a *SPECIFIC* valid cert.  Those will have to be excluded.

    I can tell you Google Apps *hates* HTTPS d&s.  Websocket applications *hate* d&s.  Anything that includes its own certificate store - like Java apps - will likely break unless you install the cert separately.

    You have to *carefully* plan your traffic to make sure things don't go belly up on you.

    I generally recommend a completely separate zone for your employees' personal devices - don't put them on your core network - employ some sort of EMM/MDM solution if they need to access core network resources, and deploy something like Sophos Endpoint on all devices.  Note - this is not a "be all and end all" description.  Again - lots of planning.

    Go back to your re-seller.  They can help.  If they can't, *FIND ONE WHO CAN* because they are doing you NO GOOD just selling you a device without anything on the back-end to help.