Publishing Exchange 2016 through WAF in XG v18 not working - HTTP 401 and 400 errors

Hi,

I have managed to solve the issue with the 401 error by enabling Keep Alive in the Webserver settings that defines Exchange as webserver. I now can successfully access OWA, EWS and Autodiscover using a webbrowser.

However, using the Microsoft Remote Connectivity Analyzer EWS test I receive the following error and the test fails:

The Sophos XG web log now shows http 400 bad request errors. The test works fine if I publish Exchange using good old Microsoft TMG. Any other ideas?

Thanks
Michael

After further testing it turnes out that the error in Microsoft Remote Connectivity Analyzer only occues if the EWS request is proxied from Exchange 2016 to Exchange 2010. If either the mailbox is directly in Exchange 2016 it works fine, or if I set the Web Server in the XG to Exchange 2010.

It however is strange that the WAF in Microsoft TMG does not cause these issues.

/edit:

I got it working correctly now. I re-read information regarding how the proxying of EWS requests from Exchange 2016 to 2010 works and found that Exchange 2016 (which receives requests to its EWS service from the Sophos XGs WAF) loggs information here:

C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Ews

https://techcommunity.microsoft.com/t5/exchange-team-blog/client-connectivity-in-an-exchange-2016-coexistence-environment/ba-p/603945

When running the Remote Connectivity Analyzer from Microsoft with the EWS test for a user that is not present directly on Exchange 2016 but on Exchange 2010, the following was logged in the aforementioned log:

2020-06-20T07:14:10.137Z,1675a684-211f-4a27-b4cb-1cab9676e32f,15,1,1913,5,,Ews,mail.example.de,/EWS/Exchange.asmx,,Negotiate,false,,,,ExchangeServicesClient/15.01.0817.000,13.74.35.9,EX03,401,,,POST,,,,,,,,,0,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,1,,1,1,,,,BeginRequest=2020-06-20T07:14:10.136Z;CorrelationID=<empty>;SharedCacheGuard=0;EndRequest=2020-06-20T07:14:10.137Z;,,,,,,
2020-06-20T07:14:10.206Z,2133b107-53f8-4c57-bac8-d82024755be0,15,1,1913,5,,Ews,mail.example.de,/EWS/Exchange.asmx,,Negotiate,true,DOMAIN\username,,Sid~S-1-5-21-1585345765-1457212253-29105016-17944,ExchangeServicesClient/15.01.0817.000,13.74.35.9,EX03,400,,BadRequest,POST,Proxy,ex01.internaldomain.dom,14.03.0123.000,IntraForest,WindowsIdentity,,,,764,,,,9,0,,0,,0,,0,0,,0,15,0,0,0,,,,,,0,1,5,1,,6,,15,15,,,,BeginRequest=2020-06-20T07:14:10.191Z;CorrelationID=<empty>;ProxyState-Run=None;DownLevelTargetRandomHashing=0/1;ClientAccessServer=EX01.internaldomain.dom;ResolveCasLatency=0;FEAuth=BEVersion-1937997947;ProxyToDownLevel=True;RoutingEntry=DatabaseGuid:7902f870-fe2b-4f39-a9b9-6b083c695767%40internaldomain.dom%40internaldomain.dom Server:EX01.example.dom+1937997947@0;BeginGetRequestStream=2020-06-20T07:14:10.203Z;OnRequestStreamReady=2020-06-20T07:14:10.203Z;ProxyState-Complete=ProxyRequestData;SharedCacheGuard=0;EndRequest=2020-06-20T07:14:10.206Z;,StreamProxy=StreamProxy-Request-None;HttpException=Cannot find the appropriate SOAP header or body.;,,|RoutingDB:7902f870-fe2b-4f39-a9b9-6b083c695767,,,CafeV1

The WAF log of the XG was showing the HTTP 400 Bad Request error as well:

I then searched for the Cannot find the appropriate SOAP header or body error that appeared in the Exchange 2016 HTTPproxy logfile and found the following articles:

https://support.microsoft.com/en-ph/help/2956962/outlook-clients-may-be-unable-to-set-oof-status-or-view-free-busy-in-a
https://ingogegenwarth.wordpress.com/2015/11/30/what-is-uploadreadaheadsize/

After changing the UploadReadAheadSize to 49152 on the Exchange Server 2016 Virtual Directoy (it was set to 0 before) and rebooting the server the error was solved. The HTTP 400 error is gone now and the Microsoft Remote Connectivity analyzer EWS test completes successfully even if the request needs to be proxied to Exchange 2010. (for Info: On the EWS directory in Exchange 2010 the value was already set to 49152 - maybe this is the cause why it worked if I specified the Exchange 2010 server as target in the WAF rule).

So in addition to enabling Keep Alive for the Exchange Web Server (in the XGs Web Server profile for it) this solved my problem and publishing EWS and OWA of Exchange 2016 though Sophos XG v18 WAF seems to work fine now. I will set the configuration to productive on Monday and report if any other issues come up.

Maybe this helps other users who run into a similar problem in the future.

BR
Michael

After further testing it turnes out that the error in Microsoft Remote Connectivity Analyzer only occues if the EWS request is proxied from Exchange 2016 to Exchange 2010. If either the mailbox is directly in Exchange 2016 it works fine, or if I set the Web Server in the XG to Exchange 2010.

It however is strange that the WAF in Microsoft TMG does not cause these issues.

/edit:

I got it working correctly now. I re-read information regarding how the proxying of EWS requests from Exchange 2016 to 2010 works and found that Exchange 2016 (which receives requests to its EWS service from the Sophos XGs WAF) loggs information here:

C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Ews

https://techcommunity.microsoft.com/t5/exchange-team-blog/client-connectivity-in-an-exchange-2016-coexistence-environment/ba-p/603945

When running the Remote Connectivity Analyzer from Microsoft with the EWS test for a user that is not present directly on Exchange 2016 but on Exchange 2010, the following was logged in the aforementioned log:

2020-06-20T07:14:10.137Z,1675a684-211f-4a27-b4cb-1cab9676e32f,15,1,1913,5,,Ews,mail.example.de,/EWS/Exchange.asmx,,Negotiate,false,,,,ExchangeServicesClient/15.01.0817.000,13.74.35.9,EX03,401,,,POST,,,,,,,,,0,,,,,,,,,,,,,,,1,,,,,,,,,,,,,,1,,1,1,,,,BeginRequest=2020-06-20T07:14:10.136Z;CorrelationID=<empty>;SharedCacheGuard=0;EndRequest=2020-06-20T07:14:10.137Z;,,,,,,
2020-06-20T07:14:10.206Z,2133b107-53f8-4c57-bac8-d82024755be0,15,1,1913,5,,Ews,mail.example.de,/EWS/Exchange.asmx,,Negotiate,true,DOMAIN\username,,Sid~S-1-5-21-1585345765-1457212253-29105016-17944,ExchangeServicesClient/15.01.0817.000,13.74.35.9,EX03,400,,BadRequest,POST,Proxy,ex01.internaldomain.dom,14.03.0123.000,IntraForest,WindowsIdentity,,,,764,,,,9,0,,0,,0,,0,0,,0,15,0,0,0,,,,,,0,1,5,1,,6,,15,15,,,,BeginRequest=2020-06-20T07:14:10.191Z;CorrelationID=<empty>;ProxyState-Run=None;DownLevelTargetRandomHashing=0/1;ClientAccessServer=EX01.internaldomain.dom;ResolveCasLatency=0;FEAuth=BEVersion-1937997947;ProxyToDownLevel=True;RoutingEntry=DatabaseGuid:7902f870-fe2b-4f39-a9b9-6b083c695767%40internaldomain.dom%40internaldomain.dom Server:EX01.example.dom+1937997947@0;BeginGetRequestStream=2020-06-20T07:14:10.203Z;OnRequestStreamReady=2020-06-20T07:14:10.203Z;ProxyState-Complete=ProxyRequestData;SharedCacheGuard=0;EndRequest=2020-06-20T07:14:10.206Z;,StreamProxy=StreamProxy-Request-None;HttpException=Cannot find the appropriate SOAP header or body.;,,|RoutingDB:7902f870-fe2b-4f39-a9b9-6b083c695767,,,CafeV1

The WAF log of the XG was showing the HTTP 400 Bad Request error as well:

I then searched for the Cannot find the appropriate SOAP header or body error that appeared in the Exchange 2016 HTTPproxy logfile and found the following articles:

https://support.microsoft.com/en-ph/help/2956962/outlook-clients-may-be-unable-to-set-oof-status-or-view-free-busy-in-a
https://ingogegenwarth.wordpress.com/2015/11/30/what-is-uploadreadaheadsize/

After changing the UploadReadAheadSize to 49152 on the Exchange Server 2016 Virtual Directoy (it was set to 0 before) and rebooting the server the error was solved. The HTTP 400 error is gone now and the Microsoft Remote Connectivity analyzer EWS test completes successfully even if the request needs to be proxied to Exchange 2010. (for Info: On the EWS directory in Exchange 2010 the value was already set to 49152 - maybe this is the cause why it worked if I specified the Exchange 2010 server as target in the WAF rule).

So in addition to enabling Keep Alive for the Exchange Web Server (in the XGs Web Server profile for it) this solved my problem and publishing EWS and OWA of Exchange 2016 though Sophos XG v18 WAF seems to work fine now. I will set the configuration to productive on Monday and report if any other issues come up.

Maybe this helps other users who run into a similar problem in the future.

BR
Michael

I am not a Exchange expert nor EWS, but heard from couple of sources, that this Test Tool (Analyzer) has some flaws. 

Exchange 2016+2019 uses MAPI. 

So after Microsoft moves from RPC over HTTP to MAPI over HTTP, likely you need to setup some things in XG, to get this working.

https://docs.microsoft.com/en-us/exchange/clients/mapi-over-http/mapi-over-http?view=exchserver-2019

You can copy paste the content of the UTM Template: https://community.sophos.com/kb/en-us/131787

Hi Luca,

thanks for your reply. I managed to solve the issue and just edited by post from above. Not sure why publishing Exchange with Microsoft TMG did not show that issue though. BTW: I now do not even need allowing Outlook Anywhere in a Protection Policy (I just set it to none as we are running Exchange Hybrid and don't want anything in between Exchange Online and our On Premises Exchange). Pre Auth is also turned off.

Best Regards
Michel