Guest User!

You are not Sophos Staff.

Parents
  • Yesterday we update XGS136 HA cluster from version SFOS 19.0.0 GA-Build317 to SFOS 19.0.1 MR-1-Build350 and all our WEB Servers(WAF Protection) stop working.

    Same thing happen when we update from SFOS 18.5 MR3 to 19 GA. Because I didn't have time to investigate problem in logs Im revert firmware back to 19GA and it still didn't work. Later Im upgrade back to 19 MR1 and investigate logs and see that is problem with WAF configuration file with "Cokie signing" function. We need disable that function on all WEB servers that service start working and then reenable it. 

    Im report this issue and open case 05574326.

    Before update all servers work normaly so is strange thing that revert back to previous image in GUI didn't wake up firewalls in working state as before upgrade!!!

    reverseproxy.log errors:

    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key
    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key
    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key

    About spam opened case I will report to support in next days if update fix spam issues or not(But probably it will be same)...

Reply
  • Yesterday we update XGS136 HA cluster from version SFOS 19.0.0 GA-Build317 to SFOS 19.0.1 MR-1-Build350 and all our WEB Servers(WAF Protection) stop working.

    Same thing happen when we update from SFOS 18.5 MR3 to 19 GA. Because I didn't have time to investigate problem in logs Im revert firmware back to 19GA and it still didn't work. Later Im upgrade back to 19 MR1 and investigate logs and see that is problem with WAF configuration file with "Cokie signing" function. We need disable that function on all WEB servers that service start working and then reenable it. 

    Im report this issue and open case 05574326.

    Before update all servers work normaly so is strange thing that revert back to previous image in GUI didn't wake up firewalls in working state as before upgrade!!!

    reverseproxy.log errors:

    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key
    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key
    AH00112: Warning: DocumentRoot [/sdisk/waffiles/e94c27e14abd78b16a3b25facdc5a436] does not exist
    AH00526: Syntax error on line 92 of /cfs/waf/reverseproxy.conf: Invalid encrypted key

    About spam opened case I will report to support in next days if update fix spam issues or not(But probably it will be same)...

Children
  • This is a known database corruption issue, development is working on a fix that is targeted for v19 MR2. In the meantime, support can apply a workaround on the system to fix the corrupted keys and get WAF operational again.

  • Upgrading from SFOS 19.0.1 MR-1-Build350 to SFOS 19.0.1 MR-1-Build365 does not fix this issue. All WAF servers again stop working and need manualy disable all cookie signing functions and after start reenable them......

  • As Attila wrote above, the fix is targeted for v19.0 MR2. Until that, support can help fixing the issue, and we're preparing a hotfix package too. Stay tuned...

  • Im read that but main problem is that this issue is not fixed for 2 releases...

    It happen first time from 18.5. -> 19 and has been reported. Then happen from 19 -> SFOS 19.0.1 MR-1-Build350 and has been again reported. And now again happen when upgrading SFOS 19.0.1 MR-1-Build350 -> SFOS 19.0.1 MR-1-Build365...

    I realy hope that it will be fixed in MR2 and not after MR2 because changing WAF policy is slow(GUI need a lot of time to save configuration) and this issue make services offline for long time(Instead of no downtime when upgrading...)