Last week we found that web proxy is dead. It happend unexpectedly, there we no changes for two weeks.
I upgraded from SFOS 17.1.1 MR-1 to SFOS 17.1.2 MR-2, but web proxy is dead again.
in GUI
Configure-Systemservice - Services for Web proxy is status Stopped
When I try to restart, I receive
XG230_WP02_SFOS 17.1.2 MR-2# service awarrenhttp:restart -ds nosync 503 Service Failed
What to do ?
Hi Ladislav,
To investigate further, take SSH to the XG and navigate to 5. Device Management> 3. Advance and execute the following commands, we need the output from:
Thanks,
df -h
Filesystem Size Used Available Use% Mounted on
rootfs 323.1M 2.4M 299.6M 1% /
df: /newroot: No such file or directory
df: /newroot/dev: No such file or directory
df: /newrootrw: No such file or directory
none 323.1M 2.4M 299.6M 1% /
none 3.9G 16.0K 3.9G 0% /dev
none 3.9G 48.1M 3.8G 1% /tmp
none 3.9G 14.6M 3.8G 0% /dev/shm
/dev/conf 385.4M 66.2M 319.2M 17% /conf
/dev/content 11.2G 381.0M 10.8G 3% /content
/dev/var 96.6G 25.8G 70.8G 27% /var
after command
service awarrenhttp:restart -ds nosynd
at 9:41
cd /var/cores
ls -larth
drwxr-xr-x 2 root 0 4.0K Dec 6 2017 .
drwxr-xr-x 98 root 0 4.0K Oct 1 09:41 ..
csc.log
nothing new
awarrenhttp.log
1538379681.664144263 [ 9339/ (nil)] acl-parsefile.c:1001 read_frm_file Processing [/static/proxy/awarrenhttp/header]
1538379681.664256421 [ 9339/ (nil)] acl-common.c:769 validate_addr Not CIDR mask; continuing as it is...
1538379681.664299209 [ 9339/ (nil)] acl-common.c:769 validate_addr Not CIDR mask; continuing as it is...
1538379681.711574958 [ 9339/ (nil)] acl-parsefile.c:531 parse_acl_line Duplicate acl name 'ALL', at lineno 79
1538379682.283884564 [ 9339/ (nil)] acl-parsefile.c:1001 read_frm_file Processing [/static/proxy/awarrenhttp/tailer]
1538379682.290300639 [ 9339/ (nil)] diskcache.c:1816 disk_cache_read fopen: /sdisk/httpcache/cacheidx: No such file or directory
1538379682.290316089 [ 9339/ (nil)] diskcache.c:224 disk_cache_zap creating cache
1538379682.330131796 [ 9353/ (nil)] diskcache.c:518 rmdir_recursive_background_func removing zapped cache root folder /sdisk/httpcache.001
1538379682.330154024 [ 9339/ (nil)] awarrenhttp.c:307 init_process Limits: threads: 2, maxconns: 18432, max fd: 110716, coredump: yes
1538379682.347417352 [ 9339/ (nil)] ssl.c:201 ssl_clear_certcache_init Fail to rename certcache (/sdisk/certcache) to (/sdisk/certcache.to_be_deleted) for removal: Directory not empty
1538379682.400993468 [ 9339/ (nil)] ssl.c:964 ssl_load_cert Failed to read file: '/conf/certificate/cacerts/d919ffd0.0'
1538379682.401000741 [ 9339/ (nil)] ssl.c:1041 ssl_prepare_chain Couldn't find Issuer certificate
1538379682.401003168 [ 9339/ (nil)] ssl.c:1221 init_portal Failed to create ssl chain
1538379682.402356762 [ 9339/ (nil)] ssl.c:964 ssl_load_cert Failed to read file: '/conf/certificate/cacerts/b35c37be.0'
1538379682.402363629 [ 9339/ (nil)] ssl.c:1041 ssl_prepare_chain Couldn't find Issuer certificate
1538379682.402365915 [ 9339/ (nil)] ssl.c:1221 init_portal Failed to create ssl chain
Are you using your own certificate authority in Web \ General Settings \ HTTPS Scanning Certificate Authority?
If so, can you switch back to SecurityAppliance_SSL_CA to see if that helps?
Are you using your own certificate in Administration \ Admin Settings \ Port Setting for Admin Console?
If so, can you switch back to Appliance Certificate to see if that helps?
You may need to delete and re-upload your CA or certificate. Note that there have been some problems with the Web Proxy properly getting the certificate chain with PKCS files. If you can upload each part of the chain separately that may help.
Sorry for a late response, I was OOO. Looking at the provided information, I don't see any reason that can cause the proxy to fail. I was suspecting an issue caused due to over used disk space but that looks normal here. Please check out Michael's response and update us.
I used SecurityAppliance_SSL_CA.
I had my own certificate for Admin console. I removed all my certificates and set only Applicance Certificate
I deleted all my certificates and CA.
I have regenerated ApplianceCertificate
But situation including logs without changes
1538716381.594498606 [22989/ (nil)] awarrenhttp.c:379 main Starting ...
1538716381.594518939 [22989/ (nil)] awarrenhttp.c:404 main reading configuration
1538716381.594521724 [22989/ (nil)] config.c:378 config_init called
1538716382.752134435 [22989/ (nil)] acl-ds-db.c:420 db_init database connection established [corporate]
1538716382.753314736 [22989/ (nil)] acl-ds-db.c:436 db_init database connection established [signature]
1538716382.753583295 [22989/ (nil)] acl-parsefile.c:1001 read_frm_file Processing [/static/proxy/awarrenhttp/header]
1538716382.753677650 [22989/ (nil)] acl-common.c:769 validate_addr Not CIDR mask; continuing as it is...
1538716382.753719260 [22989/ (nil)] acl-common.c:769 validate_addr Not CIDR mask; continuing as it is...
1538716382.759888065 [22989/ (nil)] acl-parsefile.c:531 parse_acl_line Duplicate acl name 'ALL', at lineno 79
1538716383.316219972 [22989/ (nil)] acl-parsefile.c:1001 read_frm_file Processing [/static/proxy/awarrenhttp/tailer]
1538716383.321566934 [22989/ (nil)] diskcache.c:1816 disk_cache_read fopen: /sdisk/httpcache/cacheidx: No such file or directory
1538716383.321580112 [22989/ (nil)] diskcache.c:224 disk_cache_zap creating cache
1538716383.351723775 [23012/ (nil)] diskcache.c:518 rmdir_recursive_background_func removing zapped cache root folder /sdisk/httpcache.001
1538716383.351743315 [22989/ (nil)] awarrenhttp.c:307 init_process Limits: threads: 2, maxconns: 18432, max fd: 110716, coredump: yes
1538716383.352958462 [22989/ (nil)] ssl.c:201 ssl_clear_certcache_init Fail to rename certcache (/sdisk/certcache) to (/sdisk/certcache.to_be_deleted) for removal: Directory not empty
1538716383.356199081 [22989/ (nil)] ssl.c:964 ssl_load_cert Failed to read file: '/conf/certificate/cacerts/b35c37be.0'
1538716383.356204972 [22989/ (nil)] ssl.c:1041 ssl_prepare_chain Couldn't find Issuer certificate
1538716383.356207312 [22989/ (nil)] ssl.c:1221 init_portal Failed to create ssl chain
1538716383.356364371 [22989/ (nil)] ssl.c:964 ssl_load_cert Failed to read file: '/conf/certificate/cacerts/b35c37be.0'
1538716383.356368007 [22989/ (nil)] ssl.c:1041 ssl_prepare_chain Couldn't find Issuer certificate
1538716383.356369727 [22989/ (nil)] ssl.c:1221 init_portal Failed to create ssl chain
If you do not use your own certificate does this problem go away? I want to make sure this is the issue before digging in to far.
In order for the your certificate to be used, the XG needs to have the entire certificate chain up to the certificate authority that was used to issue it. In the certificate page the column CA needs to have a green checkbox. There are known issues with using PFX that includes the certificate chain, please upload the root CA and intermediate CAs separately.
Problem has been solved.
The solution:
I reuploaded all my certificates and in
Protect - Web - General Settings - HTTPS Decryption and Scanning
It was necessary to change settings
HTTPS Scanning Certificate Authority
from SecurityAppliance-SSL-CA to Default.
Problem has been solved.
The solution:
I reuploaded all my certificates and in
Protect - Web - General Settings - HTTPS Decryption and Scanning
It was necessary to change settings
HTTPS Scanning Certificate Authority
from SecurityAppliance-SSL-CA to Default.
