I have XG 125 SFOS 18.0.5 MR-5-Build586 in around 40 users office.
Everyday it stops responding randomly. No Ping , No SSH no Internet.
I tried to capture console port logs and found these kind or message.
[45119.297653] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0] [45119.297657] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swapper/0:0] [45119.297663] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [ksoftirqd/1:16] [45119.298429] Kernel panic - not syncing: softlockup: hung tasks [45119.378730] Modules linked in: nf_conntrack_ipslb nfnetmap_queue(O) xt_masterconn xt_xfrmpolicy drbg ansi_cprng echainiv xfrm6_mode_tunnel ah4 qat_api(O) usdm_drv(O) qat_c3xxx(O) intel_qat(O) xt_addrtype vfp_firewall(O) debug_cntrs(O) nf_nat_ftp nf_conntrack_ftp xt_CT ebtable_filter ebtable_nat ebtables ip6t_MASQUERADE xt_muser xt_conntrack xt_LBS ip6table_filter iptable_filter xt_DNAT xt_SNAT nf_nat_masquerade_ipv6 xt_nat_lookup xt_UST xt_ust xt_firewall nat_rules sfos_rules_framework firewall ip_set_hash_mlmwsticky ip_set_hash_sslvpn iptable_mangle ip_set_hash_mac ip_set_hash_bw nf_conntrack_dns nf_nat_sip nf_conntrack_sip nf_nat_irc nf_conntrack_irc nf_nat_tftp nf_conntrack_tftp nf_nat_h323 nf_conntrack_h323 nf_nat_pptp nf_conntrack_pptp cfg80211 usbhid hid_generic hid ohci_pci ohci_hcd xhci_pci [45119.590564] xhci_hcd uhci_hcd ehci_pci ehci_hcd fw_handle_ngfw_notification fp2sp_api fp_notifier bonding lzo lzo_compress lzo_decompress cifs red red2 appdev nf_conntrack_netlink nf_nat_proto_gre nf_conntrack_proto_gre set_sessiontbl sessiontbl ip_gre gre ipcomp xfrm_ipcomp esp4 xfrm4_mode_transport xfrm4_mode_tunnel xfrm4_tunnel xfrm_user af_key xfrm_algo aesni_intel glue_helper aes_x86_64 crypto_simd cryptd cls_u32 act_mirred sch_ingress ifb sch_hfsc sch_leafprio sch_headprio sch_sfq sch_htb xt_MULTISET xt_MLM xt_SRCNETMAP xt_MARKROUTE xt_CONTINUE xt_LOGDROP xt_ULOG xt_TCPMSS xt_REDIRECT nf_nat_redirect ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_OUT_OUTDEV ip6t_rpfilter ipt_rpfilter ebt_nflog ebt_pkttype xt_serviceset xt_appset xt_hostset xt_pkttype xt_recent xt_state xt_status xt_cet xt_OUTDEV [45119.801979] xt_iprange xt_limit xt_hashlimit xt_tcpudp xt_multiport nf_conntrack_relate xt_IPMACFILTER xt_RANGENAT xt_VHDNAT ip_set_bitmap_vhost xt_FWSET xt_set ip_set_hash_maciface_fp ip_set_hash_ipiface_fp ip_set_bitmap_hotspotuser ip_set_hash_hotspotmac ip_set_bitmap_tlsrule ip_set_bitmap_appset ip_set_bitmap_fwrule ip_set_bitmap_ctrxss ip_set_bitmap_user sp2fp_api ip_set_bitmap_userpolicy ip_set_hash_ipuser ip_set_bitmap_service ip_set_bitmap_host ip_set_hash_ipmaciface ip_set_hash_l2mac ip_set_hash_ipmac ip_set_hash_ip ip_set arptable_filter arp_tables e1000e_nm(O) igb_nm(O) i2c_algo_bit ixgbe_nm(O) vxlan udp_tunnel ip6_udp_tunnel ptp pps_core mdio i2c_i801 i2c_dev i2c_core netmap(O) ip6table_nat nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat iptable_raw nf_nat_ipv4 xt_dscp nf_nat ip6_tables [45120.015277] ip_tables tun af_packet 8021q nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 ip6_tunnel tunnel6 sit ip_tunnel tunnel4 ppdev parport_pc parport nf_conntrack lineartable bitmap_api br_netfilter bridge nf_defrag_ipv4 ipv6 stp llc x_tables nfnetlink button evdev [last unloaded: nfnetmap_queue] [45120.096597] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G O L 4.14.38 #2 [45120.118327] Hardware name: Sophos XG/Default string, BIOS 5.13 (Z162-011) 01/22/2020 [45120.141612] task: ffff88017b0d1700 task.stack: ffffc90000080000 [45120.159428] RIP: 0010:adf_send_message+0xd/0xa0 [intel_qat] [45120.176203] RSP: 0000:ffff88017fd83ae0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff10 [45120.198962] RAX: 0000000000000101 RBX: 00000000fffffff5 RCX: ffff88003df65000 [45120.220409] RDX: ffff88016f286300 RSI: ffff88001e8e1e30 RDI: ffff880149658090 [45120.241848] RBP: ffff88017fd83b30 R08: 0000000000000000 R09: 0000000000000000 [45120.263287] R10: ffffffff81270730 R11: ffffffff81270730 R12: ffff880178701600 [45120.284724] R13: ffff88001e8e1e30 R14: ffff88017b3ff098 R15: ffff88017b3ff000 [45120.306163] FS: 0000000000000000(0000) GS:ffff88017fd80000(0000) knlGS:0000000000000000 [45120.330480] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [45120.347765] CR2: 000000000a829574 CR3: 0000000001c0a000 CR4: 00000000001406e0 [45120.369204] Call Trace: [45120.376605] <IRQ> [45120.382713] qat_crypto_unregister+0x1f51/0x2cc0 [intel_qat] [45120.399742] crypto_authenc_encrypt+0x9e/0x190 [45120.4131
This thread was automatically locked due to age.