This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Sophos XG API/CLI: Certificate in WAF rule cannot be changed through buggy or undocumentated parameters?

Hi @ all,

short story: I uploaded via API/CLI succuessfully a certificate to XG, changed it in webadmin but i have now trouble in firewall to change it in my WAF-rules. Please help me to archive this. The issue I have seems buggy or undocumented, because there are in API-Documentation no parameters for "SourceNetworks" and "ExceptionNetworks". That question should create clarity.

The field I'm going to change is the following:


long story:

I got a python-script, which allows me to upload my pfx-certificate to my XG (thanks to Sophos-team):


#!/usr/bin/env python

import requests
import urllib3
urllib3.disable_warnings()
MY_CERT = 'C:\\folder\\folder\\mycert.pfx'
APIURL = 'https://my.sophos.xg:4444/webconsole/APIController?'
APICall = """<Request><Login><Username>admin</Username><Password passwordform="plain">XG_password</Password></Login><Set operation="add"><Certificate><Action>UploadCertificate</Action><Name>test01</Name><CertificateFormat>pkcs12</CertificateFormat><CertificateFile>mycert.pfx</CertificateFile><PrivateKeyFile></PrivateKeyFile><Password>PFX_password</Password></Certificate></Set></Request>"""
XGREQUEST = {'reqxml' : (None, APICall)}
XGREQUEST ['mycert.pfx'] = ('mycert.pfx', open(MY_CERT, 'rb'), 'application/octet-stream')
r = requests.post(APIURL, files=XGREQUEST, verify=False)


 
After that script the certificate appears correctly in webbrowser.
Now I got a second python-script whichs allows me to fire pre-written XML-files against my XG. The nice part: it returns the answer from XG. Together with the apiparser.log its a nice debugging-tool (thanks to Sophos-community):


#!/usr/bin/env python

import requests
import urllib3
urllib3.disable_warnings()

xml_files = ['C:\\folder\\folder\\myFile1.xml', 'C:\\folder\\folder\\myFile2.xml']  # Add xml files here

ip_address = 'ip.of.the.XG'

def api_call(api_ip, xml_doc):
    api_url = r'https://{}:4444/webconsole/APIController?'.format(api_ip)
    payload = {'reqxml' : (None, open(xml_doc, 'rb'))}
    r = requests.post(api_url, files=payload, verify=False)
    print(r.text)

for file in xml_files:
    api_call(api_ip=ip_address, xml_doc=file)



The first XML-file changes the certificate in webadmin of the XG. The name of the cert is "test01", which comes from the upper script.:


 <Request APIVersion="1702.1">
        <Login>
        <Username>admin</Username>
        <Password passwordform="plain">XG_password</Password>
    </Login>
    <Set>
        <AdminSettings transactionid="">
            <WebAdminSettings>
                <HTTPport></HTTPport>
                <HTTPSport>4444</HTTPSport>
                <UserPortalHTTPSPort>443</UserPortalHTTPSPort>
                <Certificate>test01</Certificate>
                <PortalRedirectMode></PortalRedirectMode>
                <PortalCustomHostname></PortalCustomHostname>
            </WebAdminSettings>
        </AdminSettings>
    </Set>
</Request>



But now is coming the XML which doesn't work, I want to change in a WAF-rule my certificate which I uploaded earlier.
I think in API its a HTTPBased-policy, because the named parameter-fields are the closest to the XG-webinterace, but I don't know really.
There are many mentioned but empty fields in one handy because the apiparser.log misses them and in the other handy they are mandatory, but currently I'm still in the script building content:


<Request APIVersion="1702.1">
    <Login>
        <Username>admin</Username>
        <Password passwordform="plain">XG_password</Password>
    </Login>
    <Set>
        <SecurityPolicy transactionid="">
            <Name>name_of_my_rule</Name>
            <Description></Description>
            <Status></Status>
            <IPFamily></IPFamily>
            <isenable></isenable>
            <Position></Position>
            <PolicyType>HTTPBased</PolicyType>
            <SourceNetworks>
                <Network>something_needed_to_be_here</Network>
            </SourceNetworks>
            <ExceptionNetworks>
                <Network>something_needed_to_be_here</Network>
            </ExceptionNetworks>
            <HTTPBasedPolicy>
                <!--HTTP base policy is only applicable for IPv4-->
                <HostedAddress></HostedAddress>
                <HTTPS></HTTPS>
                <RedirectHTTP></RedirectHTTP>
                <ListenPort></ListenPort>
                <Domains>
                    <Domain />                
                </Domains>
                <ProtocolSecurity />
                <CompressionSupport></CompressionSupport>
                <RewriteHTML></RewriteHTML>
                <RewriteCookies></RewriteCookies>
                <PassHostHeader></PassHostHeader>
                <Certificate>test01</Certificate>
            </HTTPBasedPolicy>
            <IntrusionPrevention></IntrusionPrevention>
            <TrafficShapingPolicy></TrafficShapingPolicy>
        </SecurityPolicy>
    </Set>
</Request>




The following response I get if I leave the red marked parameters empty (so the white text does not exist, only the black one):
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1702.1" IPS_CAT_VER="1">
  <Login>
    <status>Authentication Successful</status>
  </Login>
  <SecurityPolicy transactionid="">
    <Status code="501">Configuration parameters validation failed.</Status>
    <InvalidParams>
      <Params>/SecurityPolicy/SourceNetworks/Network</Params>
      <Params>/SecurityPolicy/ExceptionNetworks/Network</Params>
    </InvalidParams>
  </SecurityPolicy>
</Response>
Picture of XG-LOG:

 

I get the following response if I enter any of the red marked parameters (because I do not know, what exactly I have to enter):
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1702.1" IPS_CAT_VER="1">
  <Login>
    <status>Authentication Successful</status>
  </Login>
  <SecurityPolicy transactionid="">
    <Status code="501">Configuration parameters validation failed.</Status>
    <InvalidParams/>
  </SecurityPolicy>
</Response>

Picture of XG-LOG:

 

Perhaps you may see that the logs say the same - you are right :-)
That's why I show you the XML-files and scripts ;-)

 

Please help me to change the xml correctly.

greetings



This thread was automatically locked due to age.
Parents
  • There is a current bug in the API.

    NC-42594 WAF - Configuring site paths via XML API ignores all access control settings

    Will be fixed in the next MR (V17.5 MR4). 

     

    But this is just another issue, you will run after fixing your current issue.

     

    Would suggest to use a get command to get the "current configuration". 

    If you use a get to get the current configuration, you can easily store them into a variable and use this to "simply" change the certificate to use the same parameters for a update. 

     

     

     

  • hello LuCar Toni,

    thank you for your idea with the "get" command, with this I could determine the rule in question and successfully adapt the certificate.
    However, I think I've hit a bug again because:
    the WAF-rule looses rule their group affiliation even if the originally get-request-parameters are shot back unchanged to the XG. Also the <Before> and <After> parameters do not help.

    <edit>
    After playing a bit with the behaviour of restoring one WAF-rule I found another bug: the authentication is simply forgotten when the request is restored see screenshot below.
    Also the path-specific routing-box is activated although that was not the case before.
    the behavior of the rule would change if the case with the authentication would not be...that bothers me tremendously :-)

    </edit>

    Are all mentioned behaviors known or do you need more information? Are they fixed in openbeta v0.17.5 MR4, too?

     


    Below you can see my WAF-rule XML, which I provided to XG with my python-script mentioned in my first post. I marked the parts which are ignored from XG obviously:
    <Request APIVersion="1702.1">
        <Login>
            <Username>admin</Username>
            <Password passwordform="plain">XG_password</Password>
        </Login>
        <Set>
      <SecurityPolicy transactionid="">
        <Name>WAF my.little.domain</Name>
        <Description/>
        <IPFamily>IPv4</IPFamily>
        <Status>Enable</Status>
        <Position>After</Position>
        <PolicyType>HTTPBased</PolicyType>
        <After>
          <Name>some rule name</Name>
        </After>
        <HTTPBasedPolicy>
          <HostedAddress>#Port2</HostedAddress>
          <HTTPS>Enable</HTTPS>
          <ListenPort>12345</ListenPort>
          <AccessPaths>
      <AccessPath>
        <allowed_networks>Any IPv4</allowed_networks>
        <auth_profile/>
        <backend>my.little.domain</backend>
        <be_path/>
        <hot_standby>0</hot_standby>
        <path>/</path>
        <stickysession_status>0</stickysession_status>
        <websocket_passthrough>0</websocket_passthrough>
      </AccessPath>
    </AccessPaths>
          <Exceptions>
    </Exceptions>
          <ProtocolSecurity/>
          <CompressionSupport>Disable</CompressionSupport>
          <RewriteHTML>0</RewriteHTML>
          <PassHostHeader>Disable</PassHostHeader>
          <Domains>
            <Domain>my.little.domain</Domain>
          </Domains>
          <RewriteCookies>Disable</RewriteCookies>
          <Certificate>test01</Certificate>
          <RedirectHTTP>Disable</RedirectHTTP>
        </HTTPBasedPolicy>
        <IntrusionPrevention>LAN TO WAN</IntrusionPrevention>
        <TrafficShapingPolicy>VoIP Guarantee</TrafficShapingPolicy>
      </SecurityPolicy>
        </Set>
    </Request>


    Here is the get-XML, which I provided to XG with my python-script mentioned in my first post:
    <Request APIVersion="1702.1">
            <Login>
            <Username>admin</Username>
            <Password passwordform="plain">XG_password</Password>
        </Login>
        <Get>
            <SecurityPolicy transactionid="">
            </SecurityPolicy>
        </Get>
    </Request>

     

    greetings

  • Hi, 

    As mentioned earlier, you are affected by the Issue: NC-42594, which will be resolved in MR4. 

     

    Would suggest to wait until MR4 is released couple of days from now and retest your script. 

  • Hi again,

    yesterday the MR4 hit my appliance and today I updated - what should I say?
    The ACL is kept-thanks to the whole team-great work(really!), but the WAF-rule still looses rule their group affiliation even if the originally get-request-parameters are shot back unchanged to the XG.

    Technically it works now, but not smooth enough for me. Please tell me, in which beta the issue will be fixed - thanks so far!

    If you need screenshots or scriptexamples please let me know, I'm providing them as fast as possible after my holidays.
    However you can look to my post before this one.

    greetings

Reply
  • Hi again,

    yesterday the MR4 hit my appliance and today I updated - what should I say?
    The ACL is kept-thanks to the whole team-great work(really!), but the WAF-rule still looses rule their group affiliation even if the originally get-request-parameters are shot back unchanged to the XG.

    Technically it works now, but not smooth enough for me. Please tell me, in which beta the issue will be fixed - thanks so far!

    If you need screenshots or scriptexamples please let me know, I'm providing them as fast as possible after my holidays.
    However you can look to my post before this one.

    greetings

Children
  • Hi again,

    last week the MR5 hit my appliance and I updated.

    The WAF-rule still looses rule their group affiliation even if the originally get-request-parameters (the body of SecurityPolicy-tag) are shot back unchanged to the XG.

    Please tell me, in which beta the issue will be fixed - thanks so far!

     

    greetings

  • This is a issue within the API, not WAF in particular. 

    Currently i do not have any access to some resources. 

    Please feel free to open a support case to report this bug. 

  • Hi, 

    I was wondering if there is any progress regarding this issues. I have sort of the same issue. 

    1. Retrieved the response of a HTTPBased SecurityPolicy

    2. Modified the certificate name 

    3. Posted the modified security policy 

    Resulting in response

    <Response APIVersion="1702.1" IPS_CAT_VER="1">
        <Login>
            <status>Authentication Successful</status>
        </Login>
        <SecurityPolicy transactionid="">
            <Status code="501">Configuration parameters validation failed.</Status>
            <InvalidParams />
        </SecurityPolicy>
    </Response>

    SFVUNL_HV01_SFOS 17.5.8 MR-8# cat /log/apiparser.log
    INFO      Oct 21 20:42:34 [27013]: Start Login Handler,Component : Login
    ERROR     Oct 21 20:42:34 [27013]: Key:ISCrEntity is not found in RequestMap File for Login.
    INFO      Oct 21 20:42:34 [27013]: Mapping file for Login component is /_conf/csc/IOMappingFiles//1702.1/Login/Login.xml
    ERROR     Oct 21 20:42:34 [27013]: Flag setting for this opcode is 18.
    INFO      Oct 21 20:42:35 [27013]: Opcode response: status:200
    INFO      Oct 21 20:42:35 [27013]: Authentication Successful
    INFO      Oct 21 20:42:35 [27013]: Start Set Handler,Component : SecurityPolicy
    ERROR     Oct 21 20:42:35 [27013]: Key:ISCrEntity is not found in RequestMap File for SecurityPolicy.
    ERROR     Oct 21 20:42:35 [27013]: Parser Error: xmlvalue for jsonkey="tempsourceid", xmlelement="/SecurityPolicy/SourceNetworks/Network" cannot be found in request file.
    ERROR     Oct 21 20:42:35 [27013]: Parser Error: xmlvalue for jsonkey="tempexceptionid", xmlelement="/SecurityPolicy/ExceptionNetworks/Network" cannot be found in request file.
    ERROR     Oct 21 20:42:35 [27013]: json object not found with key="tempsourceid" to handle logicaloperator.
    ERROR     Oct 21 20:42:35 [27013]: Parser Error: xmlvalue for jsonkey="sourceid", xmlelement="/SecurityPolicy/SourceNetworks/Network" cannot be found in request file.
    ERROR     Oct 21 20:42:35 [27013]: json object not found with key="tempexceptionid" to handle logicaloperator.
    ERROR     Oct 21 20:42:35 [27013]: Parser Error: xmlvalue for jsonkey="exceptionid", xmlelement="/SecurityPolicy/ExceptionNetworks/Network" cannot be found in request file.
    ERROR     Oct 21 20:42:35 [27013]: Flag setting for this opcode is 16.
    INFO      Oct 21 20:42:36 [27013]: Opcode response: status:500
    INFO      Oct 21 20:42:36 [27013]: End  SET Handler, Status : Success,  Component : SecurityPolicy, Transaction : , Operation : update.
    MESSAGE   Oct 21 20:42:36 [27013]: ENTITY 'SecurityPolicy' IMPORT Success
    INFO      Oct 21 20:42:36 [27013]: Command:/scripts/apiparser_generate_tar.sh /sdisk/api-1571683354644446.txt /sdisk/API-1571683354644446 /sdisk/APIXMLOutput/1571683354484.xml /sdisk/API-1571683354644446.tar /sdisk/API-1571683354644446.log 0 status:3
    INFO      Oct 21 20:42:36 [27013]: No need to create Tar file. Response file is /sdisk/APIXMLOutput/1571683354484.xml
    SourceNetworks and ExeptionNetworks are not part of the HTTPBased policy type. Am I doing something wrong or is this a known bug. and if so when will this be fixed?