anybody seeing issues with XG17 causing Outlook desktop client losing "sync" with Office365.

XG has always caused some kind of connectivity issues with Office 356 and windows updates, Ive been waiting for Sophos to add a method to use XML files or web db's of URL's & IP CIDRs for firewall rules destinations.  I managed to import a list of URLs i created from an XML file from MS but its only a web filter category so its not ideal as its till hitting the web proxy.  We need to import CIDR's to create hosts / host groups / or using XML files from urls really so firewall rules can be used with these as destinations and then set the filters & AV to none.

MS are updating the list of urls & ips all the time so best way would be to be pull from a web db.

JK

May I ask how you were able to import the XML file? I am trying to import an XML of web exceptions but i keep getting an error stating that it only accepts .TAR files even though the file i am trying to import is a .tar.  I have exported and tried editing directly in 7zip but that doesn`t work either. 

Hi All

Below is an extract of what i received from GES 

Office 365 traffic is legitimate traffic for that generally it is safe to bypass that traffic from AV scanning done on XG firewall.
With AV scanning applied on the firewall rule than in that case it is possible that outlook server will not be able to validate XG firewall's SSL certificate in that case we recommend to add exception for office 365 domains.

To bypass office 365 traffic from AV scanning we generally recommend to create LAN to WAN destination based plain FQDN rule for domains on which outlook client is making connection. This should be in the top position with no scanning applied, no policy applied except NAT policy default.(i.e. MASQ), source- any, service -any, destination- FQDN host group.

API feature available in XG firewall to create FQDN host and FQDN host group for office 365 domain.

1) refer below API query to create multiple FQDN host with single API query. here you can add <FQDNHost></FQNDHost> tag as mention below for remaining o365 domains before </Set> tag.

https://<Sophos IP>:<port>/webconsole/APIController?reqxml=<Request><Login><UserName>admin</UserName><Password>admin</Password></Login>
<Set Operation="add">
<FQDNHost><Name>*.aadrm.com</Name><FQDN>*.aadrm.com</FQDN></FQDNHost>
<FQDNHost><Name>*.activedirectory.windowsazure.com</Name><FQDN>*.activedirectory.windowsazure.com</FQDN></FQDNHost>
<FQDNHost><Name>*.adhybridhealth.azure.com</Name><FQDN>*.adhybridhealth.azure.com</FQDN></FQDNHost>
<FQDNHost><Name>*.azurerms.com</Name><FQDN>*.azurerms.com</FQDN></FQDNHost>
</Set>
</Request>

2) refer below API query to add those multiple FQDN hosts created in step 1) to one FQDN Hostgroup name (i.e. O365_domains)

https://<Sophos IP>:<port>/webconsole/APIController?reqxml=<Request><Login><UserName>admin</UserName><Password>admin</Password></Login>
<Set Operation="add">
<FQDNHostGroup>
<Name>O365_domains</Name>
<FQDNHostList>
<FQDNHost>*.aadrm.com</FQDNHost>
<FQDNHost>*.activedirectory.windowsazure.com</FQDNHost>
<FQDNHost>*.adhybridhealth.azure.com</FQDNHost>
</FQDNHostList>
</FQDNHostGroup>
</Set>
</Request>

3) Now you can use FQDN hostgroup O365_domains created in step 2) as destination in LAN to WAN plain firewall rule.

What they dont tell you is that there are challenges importing the FQDN Objects due to pure amount of them. (if you use the full MS List - which we are testing) You can overcome it using CURL.

Despite the result I am not convinced this is a real solution. Its a band-aid. The fact is MS will always be changing & its simply not practical to manually make these changes. 

great just what i been looking for!!! Your a star, Sophos support well web chat tech support agent said it couldnt be done.  I did tell him ive seen it before mentioned but i wasnt able to find the specifics.

anyway ill get on that now.

I have created a feature request for this problem of FQDN lists, we need a way to import FQDN lists from files and web dbs like we can with Web filter URLs.

Hopefully they will add this ASAP. as we both mentioned MS lists are always changing and recently they have changed the lists completely.  

https://techcommunity.microsoft.com/t5/Office-365-Blog/Announcing-Office-365-endpoint-categories-and-Office-365-IP/ba-p/177638

JK

I had a customer run into this issue.  The rule set did NOT have ANY filtering or scanning on it.  It was driving the customer nuts.  They are an HR outfit and they were missing critical employee emails because of the issue, which is not good when that is you reason for being in business is to answer those emails!  I went back and forth trying to trace down the issue to no avail.  They had a complete city block power outage that ran the UPSs out and shut down the Firewalls for about 5-10 minutes.  It came back up and everything is good now.  This is deeper than a exemption list.

Internally we had the MS Update issue that drive our engineer nuts for about 2 days until he finally figured out that it was FW issue.  He put in the exemptions for MS Updates and that did fix that issue.  And this was fixed in the last FW update.

So while it may seen to be the same issue, it is really two different issues.

Charles Johnson 

You can do mass imports/configurations with an XML through the API. With the XG API you can pass short parameters directly into the URL bar of your browser. For longer XML files like what we want to use here your best bet is to use the API with something like cURL or Python to pass the entire XML file. 

HERE is a link to a blog post which details the syntax for cURL and Python. The API is very quick and seems to work pretty well so far. I hope that helps. 

You have helped me a ton. I am still failing with using cURL.  I tried using postman and I guess my rookie skill set here is hindering me.  This is what i am getting back

COMPANY+it@IT-DIRECTOR MINGW64 ~
$ curl -k 10.1.70.1:4444/webconsole/APIController?-F C:\Users\blank\Des ktop\O_365_FQDN_HOST.xml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (52) Empty reply from server
curl: (3) Port number ended with 'U'

Not sure where the snag is here - Off the top of my head you should have a space on both sides of the '-F' flag.  Also - prefix the IP with https://. It Should be 

$ curl -k https://10.1.70.1:4444/webconsole/APIController? -F C:\Users\blank\Des ktop\O_365_FQDN_HOST.xml

That said I think I know what your trying to do. I actually just did the same thing yesterday and had some issues. If you are using the FQDN list that you found here there are some duplicates and issues with the format of the FQDNs which causes the API to reject the changes. 

I had to filter all the doubles out and a couple others. In total I think I pulled 6 out. 

Based on the traffic on these forums I am guessing that there are more than just you and I trying to get this done. I will try and get a tutorial posted today on my blog with a walk through and the full xml you need to add all Office 356 Hosts. I will post the link here when that is done. 

Also, I don't know how comfortable you are with Python, but if your on a windows machine may be easier to use that. If you have access to a Linux box I would use that for cURL before Windows. (That may just be my personal hatred for Windows coming out)

Charles Johnson 

Sorry, this weekend go away from me and I was not able to put the tutorial up yet. I will just post the XML I used here so you and others can have access to it. As I mentioned before, this is adapted from the Office365 zip file from Sophos support. I changed all domains to have the wild card '*', then wrapped them in xml tags. The other major difference was that I had to remove six domains from the list because they were causing the API to error out. 

These two attached files contain the XML to add all the domain objects from the O365 list (minus the six bad ones) then all those objects are added to a host group. 

First, send the FQDN_HOSTS.xml file to the API (Creates the objects)

Then, send the FQDN_GROUP.xml to the API (Assigns all objects to FQDN Group)

You can send me a private message if it is still not working and we can troubleshoot it further. 

7026.FQDN_HOSTS.xml0383.FQDN_GROUP.xml

To add some context to this post that Adam has shared:

This information was provided by our support to assist with Adam's specific support case and issue. Use the information above at your own discretion. If you had any questions regarding the information and suggestion, I would advise to please contact support for further clarification and confirmation.

Regards,

Flo

Hi All

Below is the update received today (after we had a productive meeting with our appointed Sophos Sales Engineer). We have conveyed some concerns with the blanket approach however as the email suggests its interim whilst the issue is raised with product management. In our eyes this issue goes well beyond just the initial "support" help we have a problem... but more for development / product management teams to factor in a simple yet manageable solution for products such as Office 365.

Cheers

Adam

Sent: Wednesday, 23 May 2018 5:26 PM
Subject: Office 365 domains

Hello Adam, Garth

Following on from our call on Monday, this is the list of domains used by our Pro Services in Office 365 environments, derived from the Microsoft article.  I suspect that there is much on this list that doesn’t actually need to be there for our bypass purposes.

*.cloudappsecurity.com

*.onmicrosoft.com

*.office.net

*.office.com

*.office365.com

*.microsoft.com

*.microsoftonline.com

*.live.com

*.azure.net

*.msecnd.net

*.windows.net

*.windowsazure.com

*.sharepointonline.com

*.visualstudio.com

*.cloudapp.net

*.azureedge.net

I have attached an tarr’ed XML file that you can import.  You will find an FQDN Group that I called “Office 365 Domains”.  You can add an Office 365 as an allowed exception at the top or near the top of your firewall rules.  Your rule would look something like this:

You can import the XML by going Backup & Firmware -> Import Export.   In my experience, using import export can impact performance, particularly on the smaller units, so I wouldn’t import it on a live system.   The log viewer “admin” view will show success of import.

As discussed, this is to help in the near term.  I’ll let you know of any feedback I get from our product management…

Regards"

Hi All

Below is the update received today (after we had a productive meeting with our appointed Sophos Sales Engineer). We have conveyed some concerns with the blanket approach however as the email suggests its interim whilst the issue is raised with product management. In our eyes this issue goes well beyond just the initial "support" help we have a problem... but more for development / product management teams to factor in a simple yet manageable solution for products such as Office 365.

Cheers

Adam

Sent: Wednesday, 23 May 2018 5:26 PM
Subject: Office 365 domains

Hello Adam, Garth

Following on from our call on Monday, this is the list of domains used by our Pro Services in Office 365 environments, derived from the Microsoft article.  I suspect that there is much on this list that doesn’t actually need to be there for our bypass purposes.

*.cloudappsecurity.com

*.onmicrosoft.com

*.office.net

*.office.com

*.office365.com

*.microsoft.com

*.microsoftonline.com

*.live.com

*.azure.net

*.msecnd.net

*.windows.net

*.windowsazure.com

*.sharepointonline.com

*.visualstudio.com

*.cloudapp.net

*.azureedge.net

I have attached an tarr’ed XML file that you can import.  You will find an FQDN Group that I called “Office 365 Domains”.  You can add an Office 365 as an allowed exception at the top or near the top of your firewall rules.  Your rule would look something like this:

You can import the XML by going Backup & Firmware -> Import Export.   In my experience, using import export can impact performance, particularly on the smaller units, so I wouldn’t import it on a live system.   The log viewer “admin” view will show success of import.

As discussed, this is to help in the near term.  I’ll let you know of any feedback I get from our product management…

Regards"

I want to note that I have now imported the full list of 230+ FQDNs from the list circulating around here; and it did NOT resolve the issue. My clients are still reporting the sync issue. I have a ticket with support but I am still fighting my way up the escalation ladder. 

It seems the approach with adding the massive FQDN list is to just throw everything at it and hope something works. This is disappointing to say the least. 

Joe

Care to share how you have this setup? I am curious as so far across many sites we are now functioning. (Note: we are still slowing progressing our changes through the rest)

I am sure many here are happy to assist

Cheers

Adam

Adam, 

I am not sure by what you mean as setup? I imported all the FQDNs with the API - then created a firewall rule which turns off all scanning to those domains. I have also imported all of the URLs as exceptions - neither change fixed the issue. 

Joe

Simple screenshot would suffice (just 2nd set of eyes) , however it does sound like you have it setup correct.

I assume you have the firewall rule "top" ?

What doe the the sophos logs say?

Have you put a wireshark on one of the impacted machines? If so did this uncover anything?

I assume you have ruled out all other areas, eg: DNS ?

Adam

The logs say nothing, and after working with support and reviewing the forums I assume this is because it is more of a bug than a feature of the firewall.

I have done packet captures, reviewed them and sent them to support - they didn't see the issue either. 

Adam - Are you saying you were able to fix this issue? 

Joe

Honestly too early to say if the issue has 100% gone. The one of two locations i was using as a test case with GES Support the issue has definitely subsided since the rules have been setup. The other we are still monitoring. We have 20+ XG's which have exhibited the behavior. As mentioned a little earlier today we received an update from our Sophos Sales Contact who has been willing to listen to the issue. I posted his response which also contains what apparently is recommended by Sophos Pro Services. Tonight i implemented this on another XG so will need to monitor it to see how it goes.

Happy to share the file with you if you would like to give it a try - just DM your contact details

Cheers

Adam

I have to agree with Joe.  We have also done this and I am still having users with sync issues.  The FQDN list is not a fix.  There were a few times I thought it was working for a few days, but nope.  There is something fundamentally wrong with the firmware or something.  From what I can tell anyways...   

Not for a second do i doubt what you both are saying. This issue has been so intermittent its quite difficult to establish any sense of reason.

One thing i would suggest though for troubleshooting purposes anyway... is pick a single machine which regularly exhibits the behavior & give it a basic LAN > WAN rule (no features etc & for any destination & any service) see how this goes. 

Just thinking out loud 

Adam, 

That was my first troubleshooting step. I completely white listed one host hoping that would resolve the issue. But unfortunately there is something fundamentally flawed in the firmware. 

I will throw in my 2 cents,  I have only had one customer that had this issue.  It was driving me nuts.  I had the rules set to any any and no restrictions any were.  That was when I thought I had ruled out the XG as the issue, but I could not figure what was going.   I was going to reset switches just to see if the switches where having sometime of weird ARP issue.  I was guessing at things at this point.  They had a power outage that lasted longer than the battery backup and shut down everything.  When it came back up, everything was fixed.  I would love to see someone do a power rest of the XG and see if that fixes the issue.   They were using a REV1 XG135 in HA Active-Passive mode.