Guest User!

You are not Sophos Staff.

Thread Info
  • Locked Locked
  • Replies 39 replies
  • Subscribers 21 subscribers
  • Views 185342 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Memory Leak in Server 2008R2/2012R2/2016

Matthew Gustke

 We have identified a slow memory leak on servers that have Sophos installed.  We have used Task Manager, ProcMon, Process Explorer, Vmmap and RamMap to try to isolate and identify the process(es) that is consuming the nonpaged pool of memory.  So far our diagnostics have not given us any indication at all about what is consuming and not releasing the nonpaged pool. 

 

We have restarted all Sophos processes that can be restarted and it did not free up any of the nonpaged pool.  Three Sophos processes are unable to be restarted (access denied), so we have no way to tell if they are holding on to the nonpaged pool.

 

We have turned off tamper protection in Sophos and disabled all features and it did not free up any of the nonpaged pool.

 

Five identical servers had Sophos installed and were having issues with the memory leak.  We removed Sophos from one server and it is functioning normally now, while the other 4 continue to have the issue.  We have done the same thing with two other pairs of identical servers and we have had identical results.  Removing Sophos clears up the slow memory leak.  Installing Sophos on the servers causes the memory leak to return.

 

On servers that become non-responsive due to the memory leak, the only solution has been to do a hard-reboot, which clears the nonpaged pool until the leak fills it back up again.

When viewing task manager, the memory consumed by the list of processes does not add up to the total memory usage.  The culprit is the massive amount of Nonpaged Kernel Memory that is being consumed.

 

These servers are running the latest version of Sophos.

 



This thread was automatically locked due to age.
  • in reply to Matt Collier

    Hi Matt,

    Have you rebooted the server since you updated? The driver will not be upgraded until a reboot has taken place.

    Regards,

    Stephen

  • in reply to StephenMcKay

    Stephen,

     

    It's possible some of these servers had not rebooted since the update, but I'm wanting to know what specific version we need to be looking for? We have 1400 server deployments, and we would like to be able to use the central console to ensure all devices are on the correct version but that isn't possible. Is there a version number we could look for via programs and features? We could run a report from our RMM toolset.

     

    Thanks,

    Matt

  • in reply to Matt Collier

    Hi Matt,

    Let me check with the team the best way to do this, the product version will show Server Core 2.0.1, but the SED driver will not load until a reboot has completed. The version of SED should be 1.5.0.59.

    Could you check for servers with PendingRenameFileOperations? 

    Regards,

    Stephen

  • in reply to StephenMcKay

    Hi Stephen,

     

    We were instructed to do the following on the servers as a work around:

    1)      Turn off Tamper Protection

    2)      Open Regedit

    3)      Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Sophos Endpoint Defense\

    4)      Open “DebugLevel” and set it to 8.

    5)      Restart the system

     

    Do we need to revert these changes after the new version or is it OK to continue with this value set to "8"?

     

    Regards,

    Henrik

  • in reply to Henrik Witoslaw

    Hi Henrik,

    You can reset your DebugLevel back to 2. I don't think this was a workaround, more that we wanted to get more detailed logging prior to you sending us an SDU. 

    Regards,

    Stephen

  • Can a confirmation be made on whether this has been resolved for people who went straight to 2.0.2 core agent from the affected version? We had to as we were on the affected version due to an issue with update cache's not pulling the correct update ( I originally had planned to skip the version that had this problem entirely and go to 2.0.1). We upgraded an affected server to 2.0.2, however the behavior seems to be similar to what was taking place when leaks would occur. Wanted to see if the update actually did solve the problem for those that were affected as well. 

  • in reply to Steven Brook

    Hi Steven,

    We had reports that upgrades did resolve this for affected customers; we also run soak tests and look for leaks during all phases of our testing; again we have seen none in these latest releases. 

    Regards,

    Stephen

  • Hi there,

     

    please can you confirm if this memory leak issue also affects Sophos Endpoint Security and Control v10.8?

    We currently have a mixture of Windows Server 2008 & 2008 R2 file servers, most of which are exhibiting high memory usage.

    thanks

  • I'm currently experiencing what looks to be the same issue with a client.  They have about 40 endpoints deployed,  all Windows 10 Pro.  About 5 or 6 systems are currently affected out of the bunch.  Non paged pool slowly fills up over the course of about a week,  reboot cures it temporarily.  Removing the endpoint stops it completely.

Unfiltered HTML