This discussion has been locked.

You can no longer post new replies to this discussion. If you have a question you can start a new discussion

XG450 HA 18.0.3 MR-3 to SFOS 18.5.1 MR-1-Build326 works but automatically rolled back

Today we upgraded a XG450 HA Cluster from 18.0.3 to 18.5.1

Everything seemed to work fined, both appliances showed 18.5.1 on the display.

after we reloaded the cluster, both appliances reverted back to 18.0.3.

from the GUI i could see that 18.5.1 was installed in the 2nd partition but was not booted.

then I tried to boot it manually, by hitting the boot firmware button, which caused the cluster to relead with 18.5.1 but after a few minutes reload again with 18.0.3

same result when hitting the "download&install" button.

which logs to check?

This thread was automatically locked due to age.

Parents

0 LuCar Toni over 3 years ago

Check the migration.log on the firewall.

This log is mainly responsible to get the firmware configration running. If this fails, it will rollback to MR3.

if there is an problem with a certain object /config you could fix this or wait for V18.5 MR2 and assume it would be already fixed.
Cancel
Vote Up 0 Vote Down

Cancel
0 Samuel Heinrich over 3 years ago in reply to LuCar Toni

below is the log of both appliances.

As far as I can tell, the aux appliance was the one causing the problem, since I was able to login into the primary for a short time.

**********************************************
**********************************************
**********************************************

Primary Appliance

**********************************************
**********************************************
**********************************************
...
Sat Nov 13 07:00:37 UTC 2021 starting old version corporate db
Starting conf database
381 2021-11-13 07:00:37.895 GMTLOG: could not connect socket for statistics collector: Network is unreachable
381 2021-11-13 07:00:37.895 GMTLOG: disabling statistics collector for lack of working socket
383 2021-11-13 07:00:37.895 GMTLOG: database system was shut down at 2021-11-13 06:58:06 GMT
381 2021-11-13 07:00:37.898 GMTLOG: database system is ready to accept connections
2021-11-13 08:00:39.813657+01
Sat Nov 13 07:00:39 UTC 2021 : Database started after 0 seconds
DROP SCHEMA
UPDATE 3
Stopping database
381 2021-11-13 07:00:41.313 GMTLOG: received fast shutdown request
381 2021-11-13 07:00:41.313 GMTLOG: aborting any active transactions
384 2021-11-13 07:00:41.314 GMTLOG: shutting down
384 2021-11-13 07:00:41.726 GMTLOG: database system is shut down
Sat Nov 13 07:00:42 UTC 2021 : Database stopped after 1 seconds
/sdisk/oldpgconfdump.sql is created
Starting conf database
426 2021-11-13 07:00:43.032 GMTLOG: could not connect socket for statistics collector: Network is unreachable
426 2021-11-13 07:00:43.032 GMTLOG: disabling statistics collector for lack of working socket
428 2021-11-13 07:00:43.033 GMTLOG: database system was shut down at 2021-07-27 19:07:30 GMT
426 2021-11-13 07:00:43.037 GMTLOG: database system is ready to accept connections
2021-11-13 07:00:45.023007+00
Sat Nov 13 07:00:45 UTC 2021 : Database started after 0 seconds
DROP SCHEMA config CASCADE
DROP SCHEMA
DROP SCHEMA public CASCADE
DROP SCHEMA
DROP PROCEDURAL LANGUAGE plpgsql
440 2021-11-13 07:00:46.586 GMTERROR: cannot drop language plpgsql because extension plpgsql requires it
440 2021-11-13 07:00:46.586 GMTHINT: You can drop extension plpgsql instead.
440 2021-11-13 07:00:46.586 GMTSTATEMENT: DROP PROCEDURAL LANGUAGE plpgsql
ERROR: cannot drop language plpgsql because extension plpgsql requires it
HINT: You can drop extension plpgsql instead.
CREATE SCHEMA public
CREATE SCHEMA
psql:/sdisk/oldpgconfdump.sql:18483: WARNING: column "senderemail" has type "unknown"
DETAIL: Proceeding with relation creation anyway.
psql:/sdisk/oldpgconfdump.sql:18483: WARNING: column "receipientemail" has type "unknown"
DETAIL: Proceeding with relation creation anyway.
setval
--------
179
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
92
(1 row)

429 2021-11-13 07:00:49.293 GMTLOG: checkpoints are occurring too frequently (6 seconds apart)
429 2021-11-13 07:00:49.293 GMTHINT: Consider increasing the configuration parameter "checkpoint_segments".
Stopping database
426 2021-11-13 07:00:52.665 GMTLOG: received fast shutdown request
426 2021-11-13 07:00:52.665 GMTLOG: aborting any active transactions
429 2021-11-13 07:00:53.147 GMTLOG: shutting down
429 2021-11-13 07:00:53.278 GMTLOG: database system is shut down
Sat Nov 13 07:00:53 UTC 2021 : Database stopped after 1 seconds
old conf to new conf migrated with return value :: 0
Sat Nov 13 07:00:53 UTC 2021 starting migration log
Starting conf database
490 2021-11-13 07:00:53.949 GMTLOG: could not connect socket for statistics collector: Network is unreachable
490 2021-11-13 07:00:53.949 GMTLOG: disabling statistics collector for lack of working socket
492 2021-11-13 07:00:53.949 GMTLOG: database system was shut down at 2021-11-13 07:00:53 GMT
490 2021-11-13 07:00:53.952 GMTLOG: database system is ready to accept connections
2021-11-13 07:00:55.940105+00
Sat Nov 13 07:00:55 UTC 2021 : Database started after 0 seconds
INSERT 0 0
UPDATE 1
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
UPDATE 1
INSERT 0 1
UPDATE 1
nvram_get failed with -16
Old version is 18.016 and currentversion is 18.507
Database is upgrading to dbv18.030
Check migration for version dbv18.030
Applying migration for version dbv18.030
Database is upgrading to dbv18.031
Check migration for version dbv18.031
Applying migration for version dbv18.031
Database is upgrading to dbv18.032
Check migration for version dbv18.032
Applying migration for version dbv18.032
remove_duplicate_entry
------------------------

(1 row)

Database is upgrading to dbv18.033
Check migration for version dbv18.033
Applying migration for version dbv18.033
add_missing_log_settings_ha
-----------------------------

(1 row)

Database is upgrading to dbv18.034
Check migration for version dbv18.034
Applying migration for version dbv18.034
Database is upgrading to dbv18.500
Check migration for version dbv18.500
Applying migration for version dbv18.500
Database is upgrading to dbv18.501
Check migration for version dbv18.501
Applying migration for version dbv18.501
Database is upgrading to dbv18.502
Check migration for version dbv18.502
Applying migration for version dbv18.502
Database is upgrading to dbv18.503
Check migration for version dbv18.503
Applying migration for version dbv18.503
add_columns
-------------

(1 row)

Database is upgrading to dbv18.504
Check migration for version dbv18.504
Applying migration for version dbv18.504
add_missing_log_settings_ha
-----------------------------

(1 row)

Database is upgrading to dbv18.505
Check migration for version dbv18.505
Applying migration for version dbv18.505
Database is upgrading to dbv18.506
Check migration for version dbv18.506
Applying migration for version dbv18.506
add_columns
-------------

(1 row)

remove_duplicate_entry
------------------------

(1 row)

Database is upgrading to dbv18.507
Check migration for version dbv18.507
Applying migration for version dbv18.507
create_pguser
---------------

(1 row)

create_default_snat_rule
--------------------------

(1 row)

UPDATE 1
nvram_get failed with -16
Wizard restore = and initial setup value = 0
Stopping database
490 2021-11-13 07:01:00.297 GMTLOG: received fast shutdown request
490 2021-11-13 07:01:00.297 GMTLOG: aborting any active transactions
493 2021-11-13 07:01:00.298 GMTLOG: shutting down
493 2021-11-13 07:01:00.502 GMTLOG: database system is shut down
Sat Nov 13 08:01:01 CET 2021 : Database stopped after 1 seconds
applymigration.sh exited with 0
Doing /conf/certificate/cacerts/
WARNING: Skipping expired Certificate Sigen_CA.pem
WARNING: Skipping duplicate certificate Go_Daddy_Root_Certificate_Authority_G2.pem
WARNING: Skipping expired Certificate Autoridade_Certificadora_Raiz_Brasileira_v1.pem
Sat Nov 13 08:01:03 CET 2021: Before mountconf unmount
XG450_WP02_SFOS 18.0.3 MR-3#
XG450_WP02_SFOS 18.0.3 MR-3#
XG450_WP02_SFOS 18.0.3 MR-3#
XG450_WP02_SFOS 18.0.3 MR-3#

**********************************************
**********************************************
**********************************************

Auxilary Appliance

**********************************************
**********************************************
**********************************************

XG450_WP02_SFOS 18.0.3 MR-3# ssh admin@192.0.2.2
admin@192.0.2.2's password:

....
Sat Nov 13 07:00:37 UTC 2021 starting old version corporate db
Starting conf database
381 2021-11-13 07:00:37.895 GMTLOG: could not connect socket for statistics collector: Network is unreachable
381 2021-11-13 07:00:37.895 GMTLOG: disabling statistics collector for lack of working socket
383 2021-11-13 07:00:37.895 GMTLOG: database system was shut down at 2021-11-13 06:58:06 GMT
381 2021-11-13 07:00:37.898 GMTLOG: database system is ready to accept connections
2021-11-13 08:00:39.813657+01
Sat Nov 13 07:00:39 UTC 2021 : Database started after 0 seconds
DROP SCHEMA
UPDATE 3
Stopping database
381 2021-11-13 07:00:41.313 GMTLOG: received fast shutdown request
381 2021-11-13 07:00:41.313 GMTLOG: aborting any active transactions
384 2021-11-13 07:00:41.314 GMTLOG: shutting down
384 2021-11-13 07:00:41.726 GMTLOG: database system is shut down
Sat Nov 13 07:00:42 UTC 2021 : Database stopped after 1 seconds
/sdisk/oldpgconfdump.sql is created
Starting conf database
426 2021-11-13 07:00:43.032 GMTLOG: could not connect socket for statistics collector: Network is unreachable
426 2021-11-13 07:00:43.032 GMTLOG: disabling statistics collector for lack of working socket
428 2021-11-13 07:00:43.033 GMTLOG: database system was shut down at 2021-07-27 19:07:30 GMT
426 2021-11-13 07:00:43.037 GMTLOG: database system is ready to accept connections
2021-11-13 07:00:45.023007+00
Sat Nov 13 07:00:45 UTC 2021 : Database started after 0 seconds
DROP SCHEMA config CASCADE
DROP SCHEMA
DROP SCHEMA public CASCADE
DROP SCHEMA
DROP PROCEDURAL LANGUAGE plpgsql
440 2021-11-13 07:00:46.586 GMTERROR: cannot drop language plpgsql because extension plpgsql requires it
440 2021-11-13 07:00:46.586 GMTHINT: You can drop extension plpgsql instead.
440 2021-11-13 07:00:46.586 GMTSTATEMENT: DROP PROCEDURAL LANGUAGE plpgsql
ERROR: cannot drop language plpgsql because extension plpgsql requires it
HINT: You can drop extension plpgsql instead.
CREATE SCHEMA public
CREATE SCHEMA
psql:/sdisk/oldpgconfdump.sql:18483: WARNING: column "senderemail" has type "unknown"
DETAIL: Proceeding with relation creation anyway.
psql:/sdisk/oldpgconfdump.sql:18483: WARNING: column "receipientemail" has type "unknown"
DETAIL: Proceeding with relation creation anyway.
setval
--------
179
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
92
(1 row)

429 2021-11-13 07:00:49.293 GMTLOG: checkpoints are occurring too frequently (6 seconds apart)
429 2021-11-13 07:00:49.293 GMTHINT: Consider increasing the configuration parameter "checkpoint_segments".
Stopping database
426 2021-11-13 07:00:52.665 GMTLOG: received fast shutdown request
426 2021-11-13 07:00:52.665 GMTLOG: aborting any active transactions
429 2021-11-13 07:00:53.147 GMTLOG: shutting down
429 2021-11-13 07:00:53.278 GMTLOG: database system is shut down
Sat Nov 13 07:00:53 UTC 2021 : Database stopped after 1 seconds
old conf to new conf migrated with return value :: 0
Sat Nov 13 07:00:53 UTC 2021 starting migration log
Starting conf database
490 2021-11-13 07:00:53.949 GMTLOG: could not connect socket for statistics collector: Network is unreachable
490 2021-11-13 07:00:53.949 GMTLOG: disabling statistics collector for lack of working socket
492 2021-11-13 07:00:53.949 GMTLOG: database system was shut down at 2021-11-13 07:00:53 GMT
490 2021-11-13 07:00:53.952 GMTLOG: database system is ready to accept connections
2021-11-13 07:00:55.940105+00
Sat Nov 13 07:00:55 UTC 2021 : Database started after 0 seconds
INSERT 0 0
UPDATE 1
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
UPDATE 1
INSERT 0 1
UPDATE 1
nvram_get failed with -16
Old version is 18.016 and currentversion is 18.507
Database is upgrading to dbv18.030
Check migration for version dbv18.030
Applying migration for version dbv18.030
Database is upgrading to dbv18.031
Check migration for version dbv18.031
Applying migration for version dbv18.031
Database is upgrading to dbv18.032
Check migration for version dbv18.032
Applying migration for version dbv18.032
remove_duplicate_entry
------------------------

(1 row)

Database is upgrading to dbv18.033
Check migration for version dbv18.033
Applying migration for version dbv18.033
add_missing_log_settings_ha
-----------------------------

(1 row)

Database is upgrading to dbv18.034
Check migration for version dbv18.034
Applying migration for version dbv18.034
Database is upgrading to dbv18.500
Check migration for version dbv18.500
Applying migration for version dbv18.500
Database is upgrading to dbv18.501
Check migration for version dbv18.501
Applying migration for version dbv18.501
Database is upgrading to dbv18.502
Check migration for version dbv18.502
Applying migration for version dbv18.502
Database is upgrading to dbv18.503
Check migration for version dbv18.503
Applying migration for version dbv18.503
add_columns
-------------

(1 row)

Database is upgrading to dbv18.504
Check migration for version dbv18.504
Applying migration for version dbv18.504
add_missing_log_settings_ha
-----------------------------

(1 row)

Database is upgrading to dbv18.505
Check migration for version dbv18.505
Applying migration for version dbv18.505
Database is upgrading to dbv18.506
Check migration for version dbv18.506
Applying migration for version dbv18.506
add_columns
-------------

(1 row)

remove_duplicate_entry
------------------------

(1 row)

Database is upgrading to dbv18.507
Check migration for version dbv18.507
Applying migration for version dbv18.507
create_pguser
---------------

(1 row)

create_default_snat_rule
--------------------------

(1 row)

UPDATE 1
nvram_get failed with -16
Wizard restore = and initial setup value = 0
Stopping database
490 2021-11-13 07:01:00.297 GMTLOG: received fast shutdown request
490 2021-11-13 07:01:00.297 GMTLOG: aborting any active transactions
493 2021-11-13 07:01:00.298 GMTLOG: shutting down
493 2021-11-13 07:01:00.502 GMTLOG: database system is shut down
Sat Nov 13 08:01:01 CET 2021 : Database stopped after 1 seconds
applymigration.sh exited with 0
Doing /conf/certificate/cacerts/
WARNING: Skipping expired Certificate Sigen_CA.pem
WARNING: Skipping duplicate certificate Go_Daddy_Root_Certificate_Authority_G2.pem
WARNING: Skipping expired Certificate Autoridade_Certificadora_Raiz_Brasileira_v1.pem
Sat Nov 13 08:01:03 CET 2021: Before mountconf unmount
XG450_WP02_SFOS 18.0.3 MR-3#
Cancel
Vote Up 0 Vote Down

Cancel
0 LuCar Toni over 3 years ago in reply to Samuel Heinrich

Check the logs of the Aux.
Cancel
Vote Up 0 Vote Down

Cancel
0 Samuel Heinrich over 3 years ago in reply to LuCar Toni

The aux log is attached above in the reply.

I can not see the root cause.
Cancel
Vote Up 0 Vote Down

Cancel
0 Samuel Heinrich over 3 years ago in reply to LuCar Toni

What is your recommendation. do you think that issue will resolve it self with the next release ?
Cancel
Vote Up 0 Vote Down

Cancel
0 LuCar Toni over 3 years ago in reply to Samuel Heinrich

Would recommend a support case.
Cancel
Vote Up 0 Vote Down

Cancel

Reply

0 LuCar Toni over 3 years ago in reply to Samuel Heinrich

Would recommend a support case.
Cancel
Vote Up 0 Vote Down

Cancel

Children

No Data