Guest User!

You are not Sophos Staff.

[8.270][NOTABUG][CLOSED] Services restarting

Beginning a little over 9 hours ago, I got the first of 42 messages:

12:28 Middleware not running - restarted
12:28 IM/P2P classifier not running - restarted
12:30 Dhcpd not running - restarted
01:04 Spam filter cannot query database servers
01:29 Dhcpd not running - restarted
01:31 Middleware not running - restarted
[...]
02:39 Service Monitor not running - restarted
[...]
04:22 ACC device agent not running - restarted
[...]
08:41 IM/P2P classifier not running - restarted



After that my laptop could not connect via any VPN, nor could I get answers to ping.  The same was true with my iPhone via the cellular network.

Cheers - Bob
Parents
  • I noticed that the problems began, in each case, shortly after the Executive Report was sent.  Here are extracts from the System logs for these days:

    The 18th (yes, the lines from the 17th are in the file in the Astaro, and I did not modify it):
    2011:11:18-00:20:52 astaro report_render[2215]: Daily Executive Report
    
    2011:11:17-19:33:36 astaro rsyslog: action 3 queue[DA]: size=0 enqueued=6514 full=5409 maxqsize=1253 
    2011:11:17-19:33:36 astaro rsyslog: action 3 queue: size=987 enqueued=176631 full=27573 maxqsize=1000 
    2011:11:17-19:33:36 astaro rsyslog: main Q: size=9971 enqueued=104397 full=633 maxqsize=10000 
    2011:11:18-00:22:16 astaro postgres[2294]: [2-1] LOG:  unexpected EOF on client connection
    2011:11:18-00:24:01 astaro /usr/sbin/cron[2623]: (root) CMD (/sbin/audld.plx --trigger)
    2011:11:18-00:25:01 astaro /usr/sbin/cron[2786]: (root) CMD (   /usr/local/bin/reporter/system-reporter.pl)
    2011:11:17-19:38:36 astaro rsyslog: action 3 queue[DA]: size=0 enqueued=6553 full=5448 maxqsize=1253 
    2011:11:18-00:28:11 astaro snmpd[7887]: Received TERM or STOP signal...  shutting down...


    The 21st:
    2011:11:20-00:20:58 astaro report_render[17917]: Daily Executive Report
    
    2011:11:20-00:21:21 astaro snmpd[1572]: Received TERM or STOP signal...  shutting down...
    2011:11:20-00:21:22 astaro snmpd[1572]: snmpd: send_trap: Failure in sendto (Operation not permitted)


    Finally, here are the first lines in each of the System Monitor daemon logfiles - maybe this is what the devs are looking for.

    The 18th:
    2011:11:18-00:28:23 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 69.xx.yy.44 changed state to OFFLINE"
    
    2011:11:18-00:28:23 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 69.xx.yy.44"
    2011:11:18-00:28:24 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 69.xx.yy.44 changed state to OFFLINE"
    2011:11:18-00:28:32 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 69.xx.yy.44"
    2011:11:18-00:30:10 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Exiting..."
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Starting real server checker with 3 threads"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv6 socket"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket with source 184.xx.yy.34"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4003" severity="error" sys="System" sub="loadbalancing" name="plugin_trace4: Can't resolve k.root-servers.net, using static IP address!"


    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 68.xx.yy.47 changed state to OFFLINE"
    
    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 68.xx.yy.47"
    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 68.xx.yy.47 changed state to OFFLINE"
    2011:11:20-00:21:25 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 68.xx.yy.47"
    2011:11:20-00:22:54 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Exiting..."
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Starting real server checker with 3 threads"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv6 socket"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket with source 184.xx.yy.34"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4003" severity="error" sys="System" sub="loadbalancing" name="plugin_trace4: Can't resolve f.root-servers.net, using static IP address!"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="plugin_trace4: Tracing host f.root-servers.net [192.5.5.241]"


    Cheers - Bob
     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Reply
  • I noticed that the problems began, in each case, shortly after the Executive Report was sent.  Here are extracts from the System logs for these days:

    The 18th (yes, the lines from the 17th are in the file in the Astaro, and I did not modify it):
    2011:11:18-00:20:52 astaro report_render[2215]: Daily Executive Report
    
    2011:11:17-19:33:36 astaro rsyslog: action 3 queue[DA]: size=0 enqueued=6514 full=5409 maxqsize=1253 
    2011:11:17-19:33:36 astaro rsyslog: action 3 queue: size=987 enqueued=176631 full=27573 maxqsize=1000 
    2011:11:17-19:33:36 astaro rsyslog: main Q: size=9971 enqueued=104397 full=633 maxqsize=10000 
    2011:11:18-00:22:16 astaro postgres[2294]: [2-1] LOG:  unexpected EOF on client connection
    2011:11:18-00:24:01 astaro /usr/sbin/cron[2623]: (root) CMD (/sbin/audld.plx --trigger)
    2011:11:18-00:25:01 astaro /usr/sbin/cron[2786]: (root) CMD (   /usr/local/bin/reporter/system-reporter.pl)
    2011:11:17-19:38:36 astaro rsyslog: action 3 queue[DA]: size=0 enqueued=6553 full=5448 maxqsize=1253 
    2011:11:18-00:28:11 astaro snmpd[7887]: Received TERM or STOP signal...  shutting down...


    The 21st:
    2011:11:20-00:20:58 astaro report_render[17917]: Daily Executive Report
    
    2011:11:20-00:21:21 astaro snmpd[1572]: Received TERM or STOP signal...  shutting down...
    2011:11:20-00:21:22 astaro snmpd[1572]: snmpd: send_trap: Failure in sendto (Operation not permitted)


    Finally, here are the first lines in each of the System Monitor daemon logfiles - maybe this is what the devs are looking for.

    The 18th:
    2011:11:18-00:28:23 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 69.xx.yy.44 changed state to OFFLINE"
    
    2011:11:18-00:28:23 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 69.xx.yy.44"
    2011:11:18-00:28:24 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 69.xx.yy.44 changed state to OFFLINE"
    2011:11:18-00:28:32 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 69.xx.yy.44"
    2011:11:18-00:30:10 astaro service_monitor[6547]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Exiting..."
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Starting real server checker with 3 threads"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv6 socket"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket with source 184.xx.yy.34"
    2011:11:18-00:30:10 astaro service_monitor[4896]: id="4003" severity="error" sys="System" sub="loadbalancing" name="plugin_trace4: Can't resolve k.root-servers.net, using static IP address!"


    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 68.xx.yy.47 changed state to OFFLINE"
    
    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 68.xx.yy.47"
    2011:11:20-00:21:23 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="REF_YecroutbDr ICMP 68.xx.yy.47 changed state to OFFLINE"
    2011:11:20-00:21:25 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Set Availability Group REF_YecroutbDr to 68.xx.yy.47"
    2011:11:20-00:22:54 astaro service_monitor[7048]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Exiting..."
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4000" severity="info" sys="System" sub="loadbalancing" name="Starting real server checker with 3 threads"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv6 socket"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="Open ICMPv4 socket with source 184.xx.yy.34"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4003" severity="error" sys="System" sub="loadbalancing" name="plugin_trace4: Can't resolve f.root-servers.net, using static IP address!"
    2011:11:20-00:22:54 astaro service_monitor[19924]: id="4002" severity="info" sys="System" sub="loadbalancing" name="plugin_trace4: Tracing host f.root-servers.net [192.5.5.241]"


    Cheers - Bob
     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Children
No Data