Application filter for NTP

Hi James,

Looking at your FW-rule description, I can see all the Services are Allowed. I don't see a reason why the NTP is blocked when the services are open and Application Filter tells XG to Allow All. If you suspect that Application Filter is blocking NTP services then check, Log Viewer > Application Filter; it will only log Denied Traffic. This will give you a general idea if the Application Filter is doing a false positive and we could raise a request to fix that.

Thanks

sachingurung, 

can you explain in few lines how to properly use the Application filter?

I guess that in the Application lists there are some Apps that should not be there. Can you check internally and reply here back.

Thanks

I did some testing and found the following:

• An application filter with a Default Action of "Allow" will always allow NTP, even if you have a rule in there to block NTP
• An application filter with a Default Action of "Block" will always block NTP, even if you have a rule in there to allow NTP
• When NTP is blocked, an entry appears for port 123, but it is not identified as NTP in the Application logs

I do see Application Filter blocking some non-HTTP(S) traffic, in particular port 5223 and 5228, but as a general rule, as has been repeated stated here by others, Application filtering does not do anything for non-HTTP(S) traffic.

I'm a little disappointed, but at least now I know.

thanks

James

The evidence suggests that the application filter has little to do with HTTP(S) traffic. For instance, given the following application filter...

... one would expect none of the designated protocols to get blocked. Well, except Jabber perhaps if you were using a Cisco client on port 80.  I'm not, my Jabber traffic is XMPP on port 5222 and was blocked just fine, along with LDAP and SSH which, again, are pretty unrelated to HTTP(S).  The only ones not caught by this example app filter are IRC and NTP.

Thanks,

Gary

Gary,

there is some confusion here on this thread. You should implict deny principle and allow only what is needed. Application and web filtering came out after the first SW houses used "encapsulation" on TCP/IP. They re-encapsulate packets inside the big tunnel 80/443 ports. Because the first version of Firewall were not able to scan inside the 80/443 ports, those application were allowed. Even now, if the signature on the Firewall does not exist, you will see a lot of HTTP/S traffic or undefined 80/443. So, I repeat what I already wrote, Application filters are only used to scan the "big" tunnels (80 and 443) in order to catch and block/allow packets that are encapsulated and travel within the http/s tunnel.

It is strange that inside the Application filters there are not-http/s applications, like NTP.

Michael Dunn can you add some more technical aspects/information here?

Luk,

I completely agree that a "proper" implementation should be Deny All and then poke holes as needed. That being said, there is no logical reason the firewall rules would not work in the other direction as well... allow all and block as needed. Not a good model for corporate use, but perhaps a more common practice for the home user? Regardless, I can flip the rule around to block all and allow only the identified protocols and it still has the same issue where some are identified, others are not.

As for the question of application filtering only working on HTTP(S) traffic... I agree we need an answer from Sophos. Your explanation makes sense, but just looking at the current XG model seems to indicate a different direction. Perhaps things have changed a bit since they were first implemented? If you look at Web -> Categories you will see Content Delivery, Applets, ActiveX, P2P torrents, etc. Unless Sophos is trying to maintain a URL list of every single domain that has applets and activeX content, it would seem that lots of the encapsulation inspection has moved into the Web > Categories section.  Then, take a look at what is currently in the Applications > Application List and filter on Infrastructure or Network Services categories. There are clearly lots (and lots and lots) of items in those categories which are not expected to be HTTP(S) encapsulated traffic. Additionally, look at all of the applications defined as type Client/Server. Again, not expected to be HTTP(S) encapsulated.

I'm not saying you are wrong about what the Application Filter is supposed to be used for, but XG currently looks a whole lot like it should work the way I was expecting.

Ok, first off all, this is not my area of expertise.  So take everything that I'm about to say as just my understanding and I could be wrong.

First of all, Applications are used for non-HTTP/HTTPS all the time.  This is implemented by the IPS engine doing generic packet inspection on any incoming packet on any port.  In the case of HTTPS, the detection is done by the httpproxy (since it can read inside the encrypted packets) for everything else the detection is done by IPS.  Enforcement (dropping packets) is done by IPS.  At a pure guess, non-HTTP/HTTPS is somewhere between 5-25% of the definitions (behind the scene).  Sorry, Iferrara, though you are correct in what the majority of the Applications are for, there are many that don't fall in your definition.

Secondly, you must understand what "Allow" and "Block" really mean.  "Allow" does not mean the traffic is allowed.  It means that traffic will not be blocked at this stage.

For example, think of the HTTP Proxy.  Lets say you get set Search Engine to Allow.  Now you go to your browser and try to access Google and you can't.  Why?  Because in the firewall, HTTP and HTTPS are not allowed services.  Ok, so you allow them.  You can access Google.  And suddenly you get a block page.  Why?  You were blocked on virus.  Even though your policy says "Allow Search Engines", it actually means "Don't block due to the fact it is Search Engines".  Your traffic can still be blocked for many reasons, even though it says Allow.

So lets take that to NTP and applications.  Setting an Application of NTP to Allow doesn't mean NTP is allowed by everything in the XG system.  It means NTP is not blocked by the IPS / Application Filter.  It may still be blocked by the firewall ports not being open.

So...  If NTP requires you to have the firewall port / service for NTP open - why bother having an application definition for it?  Well, what if you want to put time restrictions on when it is allowed.  Or you want to block some things with the Application Filter but still allow NTP.  I'm not sure all the use cases.  But they decided to include NTP as a detectable app.

What does this mean - if you want to globally allow NTP, do it through a firewall rule that allows the NTP port/service.  If you want more granular control, have the firewall rule allow it and then restrict it with the Application Filter.

Finally, now that you understand that "Allow" does not actually mean Allow, you should understand that a Application Filter with a default of Allow All and certain applications set to block does not open you up for any additional access or attacks.  I think the only time you want to use a Block All is if you were trying to do something like a server or IoT device lockdown.

In the same vein, 95% of the time your application filter rule will say Action Block.  The main reason for having Action Allow is to override a lower priority Block.  For example an application filter may be: Allow GMail WebChat, Block Instant Messengers, default Allow.


So that all being said - I have no idea if the NTP application definition is correct or not, works or not.  But if someone wants to test it, they should have a good understanding of how it should work (AFAIK):
1) With no application filter at all, and a matching high level firewall rule with a Service of "NTP" the NTP traffic should be allowed.  (If not, check firewall logs)
2) When (1) is satisfied, an Application Filter of Block NTP, default Allow should block the NTP traffic.
3) When (1) is satisfied, an Application Filter of Allow NTP, Block (select All), default Allow should allow the NTP traffic.
4) When (1) is satisfied, an Application Filter of Allow NTP, default Block should allow the NTP traffic.
5) With no firewall that has a service of NTP, the NTP traffic should be blocked.
6) With no firewall that has a service of NTP, and a Application Filter of Allow NTP, the NTP traffic should be blocked.

Thanks Michael for your partecipation here.

Appreciated it!

It is strange that Sophos XG is controlling some non-http/s traffic via IPS. This is make no sense and also troubleshooting scenario can be quite complicated. UTM9 did not use IPS and Application filter for blocking non-http/s traffic. I know even other Vendors do the same think like UTM9.

Sorry, but I do not understand scenario 6. If the App Filter allows NTP, latter should be allowed and not blocked. Maybe I missed something.

UTM does non-HTTP/HTTPS as well. I just checked, there are application definitions for things like DHCP.  Definitions like Clash of Clans probably (I don't know for sure) apply to the higher level ports they connect to the game servers with.

In scenario 6, the traffic is blocked by the firewall.  The application filter does not come into play.  An application filter Allow cannot override a firewall Block.

Let me rephrase 2 more ways:

1) In order for HTTPS traffic to be allowed it must be allowed by all of these systems (Firewall, Category, Certificate, Antivirus, Application) and if any one of those systems says Blocked then the traffic is blocked.  In order for NTP traffic to be allowed it must be allowed by all of these systems (Firewall, Application) and if any one of those systems says Blocked then the traffic is blocked.

2) Traffic passes through a series of filters, in order.  Each filter has the ability to block the traffic, or allow it through its own filter and on to the next filter.  A packet on port 123 (which is what NTP uses) arrives with a particular source and destination.  The firewall tries to find a firewall rule that says matches that source, destination, and port.  If no firewall rules are matched the packet is dropped.  If a firewall rule does match then that rule is selected and it looks at the configured Application Filter for that firewall rule.  The IPS then looks at the packet on port 123 and it can apply its own filter about whether that packet is NTP or not and whether to drop it or to pass through its filter.  Possibly after IPS there may be other filters.

One point in (2) that explains this specific scenario.  Lets say you have 5 rules, each of them have different Application Filters.  None of the rules apply to port 123.  Given that no firewall rule matches, how would the system even know what application filter you wanted?  You must have a matching firewall rule before you can even choose which policy that IPS applies.

James,

There is hope. I have no idea why this started working. I just know that it did.

Could it be this? 

[^o)]

James,

There is hope. I have no idea why this started working. I just know that it did.

Could it be this? 

[^o)]

It's not working for me, even with that pattern update.

I have an application filter with a default of "Allow", but with a rule that denies all applications, and NTP still gets through the firewall.

When I change that to an application filter with a default of "Deny", but with a rule that allows NTP, NTP is still blocked, and the log entry for port 123 does not identify the application as NTP.

:(

James

James, can you double check your last test? When I run the different scenarios identified by Michael (and include your scenario as well) I get the correct result on all tests except where NTP is specifically allowed on a default-deny filter. This result is similar to my testing with the LDAP protocol.

In the above FW rules, each rule has an application filter as specified:

1. No filter
2. Application Filter of Block NTP, default Allow
3. Application Filter of Allow NTP, Block (select All), default Allow
4. Application Filter of Allow NTP, default Block
5. Application Filter of Block (select All), default Allow

My testing results are as follows:

1. Allowed (correct)
2. Blocked (correct)
3. Allowed (correct)
4. Blocked (incorrect)
5. Blocked (correct)

And, just like LDAP traffic was not identified as LDAP traffic in scenario 4 in my previous tests, NTP traffic was not identified as NTP traffic in the logs when I ran scenario 4 here as well. The issue I was experiencing previously, where NTP was not handled at all by the application filter, is something I have not been able to reproduce after receiving the signature update. Did that update fix part of the issue? Or was there a flaw in my previous test? I have no idea.  But... I have repeatedly been able to replicate a bug where a default block filter with explicit allow (scenario 4) does not behave as expected.

Great testing Gary. Just when I think, I know everything that is there to learn Michael Dunn  throws a curve ball when explaining XG[:D] I am guessing you have run into a bug because the way the behavior is intended does make sense.

However, I generally don't use application rules for simple things like NTP where firewall rule gives me a lot more flexibility. Here I agree with Luk that application rules are more suitable when distinguishing for example between http downloads and http streaming for netflix specially when using QoS. You can't throttle http whereas its easy to throttle netflix using application filters.

Having said all that, this is another case where things are complicated and non intuitive because of the way firewall rules incorporate everything including appfilter/QoS/IPS. Long time ago people did some testing on UTM on behaviors of different firewall/DNAT/Proxy rules. Once you understand that behavior rules become easy to write. I suspect a similar test or KB from sophos is in order to explain some of the nuances that Michael explains in many posts throughout the forum. Because the way it is right now, there is really no guide to follow to see how http proxy affects DNS, application filters dos and don'ts and how the whole application control is dependent on intrusion protection.

Billybob, 

here on some aspects it seems we are tilting at windmills. It does not make sense and this create confusion on Application filters. So Sophos should decide if use the application filters for all APPS or use them only for HTTP/S Applications. If they proceed with the first option, the application list is small and so they should allow us to add more applications using a custom application filter like custom IPS filter or another way to capture the packets and create the application. The other way makes more sense because the rest of the Application can be allowed/blocked by Firewall rules.

Hi BillyBob,

I'm in complete agreement that something simple like NTP could (and should) be handled by a simple FW rule. And yes, your point about app rules being well suited for differentiating different types of HTTP traffic is well taken.  A counterpoint would be... Sohpos put those network protocols (and client/server and P2P) in the application list and by golly I expect them to work! [;)]

That said, it is a bit confusing. Is XG using DPI on application rules to determine the traffic type? If I allow SMTP in the app rules, will XG block SSH sent across port 25? Who knows. Perhaps we need to break "applications" into two sections... one for filtering of HTTP(S) and one for "other" with an explanation of just how deep the inspection of that "other" will be. 

But... I do see the value here. Coming from a Shorewall/IP Tables setup, I like the idea of being able to create application filter bundles that are reusable and transportable. Imagine a "base" for servers that is a "deny all" with explicitly allowed services for network management. Then (assuming the v17 release has the add-able filters) you could add on a web server filter or an LDAP filter or a DB filter and, assuming your web servers and LDAP servers and DB servers are in host groups... you can have a single FW rule for each host group that holds the composite filter. Or something like that.

Granted, this could all just be the simple case of me completely misunderstanding next-gen firewalls. [*-)]

I have worked with different firewall vendors over the years and like everything else, they come up with new terminology and fancy words. DPI (deep packet inspection) is the new word. Back about 10 years ago we called them UTM appliances. The only difference is that UTM9 does layer7 (kind of like DPI) using iptables whereas DPI firewalls lately use snort or similar. Snort was being used as intrusion detection system where it did (DPI) to see what was actually being carried in those packets and since it could distinguish between different applications, some firewall vendors started using it for application classification also. XG being one of those firewalls... hence the next gen DPI label[;)]

I suspect your telnet test to port 25 would be classified as SMTP traffic if you actually send the HELO message because at that point there is no difference between an SMTP server communication and your telnet session.

I am old fashioned and I trust the firewall rules that I create. If I need to open 10 ports to make an application work, I like to know what I did instead of the firewall using application control doing it for me. Almost reminds me of upnp... you have no idea what is going on. That is why I like the approach of firewall rule and then augment it with application filter if the application uses the same firewall port otherwise such traffic should be denied. But different people have different needs and Sophos knows a lot more about industry trends and needs of their clientele so we will get what they give us. This however doesn't mean that the product shouldn't work as advertised and you are absolutely correct that if it is there, you should expect it to work.

Well stated! For non-pet-project stuff, things I want to just plain work, I share a very similar philosophy.

But this? XG is my current "shiny new toy" and to that end I'm going to push it until it breaks and learn what I can along the way. [H]