2012:04:19-00:25:26 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-00:25:26 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-01:23:59 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-01:23:59 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-01:26:54 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-01:26:55 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-03:20:52 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-03:20:52 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-03:25:18 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-03:25:18 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-06:35:37 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-06:35:37 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-06:40:51 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-06:40:51 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-08:55:23 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-08:55:23 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-09:23:16 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-09:23:16 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-09:25:10 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-09:25:10 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-11:24:13 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-11:24:15 HOSTNAME-2 ha_daemon[5631]: id="38C1" severity="info" sys="System" sub="ha" name="Node 1 is dead, received no heart beats!" 2012:04:19-11:24:15 HOSTNAME-2 ha_daemon[5631]: id="38B5" severity="info" sys="System" sub="ha" name="Master is dead, taking over!" 2012:04:19-11:24:15 HOSTNAME-2 ha_daemon[5631]: id="38B0" severity="info" sys="System" sub="ha" name="Switching to Master mode" 2012:04:19-11:24:17 HOSTNAME-1 ha_daemon[5742]: id="38A1" severity="warn" sys="System" sub="ha" name="Another master around!" 2012:04:19-11:24:17 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Node 2 changed mode: SLAVE -> MASTER" 2012:04:19-11:24:17 HOSTNAME-1 ha_daemon[5742]: id="38A1" severity="warn" sys="System" sub="ha" name="Another master around!" 2012:04:19-11:24:17 HOSTNAME-1 ha_daemon[5742]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-11:24:16 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Activating sync process for database on node 1" 2012:04:19-11:24:16 HOSTNAME-2 slon_control[5777]: Set mode to MASTER 2012:04:19-11:24:16 HOSTNAME-2 slon_control[5777]: Killing slon reporting [14164] 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Reading cluster configuration" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38Bb" severity="info" sys="System" sub="ha" name="Going slave mode in favour of node 1 (-1387441 sec)" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38B1" severity="info" sys="System" sub="ha" name="Switching to Slave mode" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Node 1 joined with version 8.301" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38C0" severity="info" sys="System" sub="ha" name="Node 1 is alive!" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Node 1 changed state: DEAD -> ACTIVE" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Node 1 changed mode: SLAVE -> MASTER" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="cluster mode: set master id to 1" 2012:04:19-11:24:17 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-11:24:18 HOSTNAME-1 ha_daemon[5742]: id="38A1" severity="warn" sys="System" sub="ha" name="Another master around!" 2012:04:19-11:24:18 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Activating sync process for database on node 2" 2012:04:19-11:24:19 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Node 2 changed state: ACTIVE -> SYNCING" 2012:04:19-11:24:19 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Node 2 changed mode: MASTER -> SLAVE" 2012:04:19-11:24:21 HOSTNAME-1 ha_daemon[5742]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-11:24:19 HOSTNAME-2 slon_control[5777]: Starting controlled switchover from Node 1 to 2 2012:04:19-11:24:20 HOSTNAME-2 slon_control[5777]: Slonik error, process exited with value 255 2012:04:19-11:24:20 HOSTNAME-2 slon_control[5777]: Started slon process 5789 for reporting 2012:04:19-11:24:20 HOSTNAME-2 slon_control[5777]: Set mode to SLAVE 2012:04:19-11:24:20 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Deactivating sync process for database on node 1" 2012:04:19-11:24:20 HOSTNAME-2 slon_control[5777]: Skipping slony cleanup for reporting 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Initializing tinyproxy ... 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Reloading config file 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Listening on IP 0.0.0.0 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Added upstream 192.168.81.65:8080 for [default] 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Setting "Via" header to 'Astaro HA Proxy' 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Adding Port [443] to the list allowed by CONNECT 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Now running as group "nogroup". 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5766]: Now running as user "nobody". 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5869]: Creating child number 1 of 1 ... 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5869]: Finished creating all children. 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5869]: Setting the various signals. 2012:04:19-11:24:25 HOSTNAME-2 ha_proxy[5869]: Starting main loop. Accepting connections. 2012:04:19-11:24:32 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Monitoring interfaces for link beat: eth0 eth1 eth3 eth2 eth4 " 2012:04:19-11:24:48 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Reading cluster configuration" 2012:04:19-11:24:48 HOSTNAME-2 ha_proxy[5869]: Shutting down. 2012:04:19-11:24:49 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-11:25:37 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Monitoring interfaces for link beat: eth0 eth1 eth3 eth2 eth4 " 2012:04:19-11:27:10 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received no backup heartbeats from master node!" 2012:04:19-11:27:10 HOSTNAME-2 ha_daemon[5631]: id="38A1" severity="warn" sys="System" sub="ha" name="Received backup heartbeats from master node!" 2012:04:19-11:29:43 HOSTNAME-1 slon_control[5855]: Initial synchronization for node 2 finished! 2012:04:19-11:29:43 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Deactivating sync process for database on node 2" 2012:04:19-11:29:58 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Initial synchronization finished!" 2012:04:19-11:29:59 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Node 2 changed state: SYNCING -> ACTIVE" 2012:04:19-11:46:20 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="Received command to shutdown!" 2012:04:19-11:46:29 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="--- Node is disabled ---" 2012:04:19-11:46:29 HOSTNAME-2 slon_control[5777]: Exiting... 2012:04:19-11:46:29 HOSTNAME-2 slon_control[5777]: Killing slon reporting [5789] 2012:04:19-11:46:30 HOSTNAME-2 ha_daemon[5631]: id="38A0" severity="info" sys="System" sub="ha" name="HA daemon shutting down" 2012:04:19-11:46:34 HOSTNAME-1 ha_daemon[5742]: id="38C1" severity="info" sys="System" sub="ha" name="Node 2 is dead, received no heart beats!" 2012:04:19-11:46:35 HOSTNAME-1 ha_daemon[5742]: id="38A0" severity="info" sys="System" sub="ha" name="Activating sync process for database on node 2"