v18: Bug with data counting in firewall rules?

Hi cryptochrome, 

Could you please share the screenshot of the firewall rule that shows data in and out counters? I will verify if it should be like that or not and update you.

Thanks,

Hi cryptochrome, 

Could you please share the screenshot of the firewall rule that shows data in and out counters? I will verify if it should be like that or not and update you.

Thanks,

Sure, here you go:

Firewall Rule:

Corresponding NAT rule:

Firewall Rule Details:

Note that this is just one example. I am seeing the same "reversed" counter on other incoming rules. 

How do you count the data? If someone on the internet initiates the connection and transfers a lot of data, does that count as incoming or outgoing?

Hello cryptochrome,

I fully agree with you! Your conclusion is perfectly logical and correct. However, there are considerably more illogical things in XG and no one is doing anything about them. They even add others such as defining DNAT rules. This is absolutely monstrous! And how developers solved the illogical creation of DNAT rules:

- added Server access assistant (DNAT) which automatically creates loopback and reflexive rules (I never needed to define them in my life, probably some Cyberoam paranoia)
- as public IP addresses offer IP addresses from the internal network, is it really a big problem to select only IP addresses defined in the WAN or DMZ zone, really?
- it is not possible to define the network service to which traffic is to be redirected under this unique tool. So you need to stop creating the DNAT rule, define the needed service and start over. I guess I didn't mention that it is not possible to use groups of services, but who would need it, right?
- and what a surprise, as the source network we are offered the whole world, all physical ports, internal networks, DNS domains, DNS hosts but it's really special, zones are not available. Well, who needs it, right?
- and in the final you will see an overview of all the horrors and what a surprise it is not possible to save the rule as inactive! What a surprise when a firewall rule can be saved as inactive ?!

And the biggest surprise at the end, this is GA version !!!

Sorry, this post is absolutely unrelated to this thread, I just wanted to point out that illogicality is many times more across the XG Firewall.

Regards

alda

Luca, thanks for explaining again in detail. I fully understand how it works, and I completely understand where you see the problem:

From the LAN Interface, the Connection could be Logged as IN/OUT.
From the WAN Interface, the same Connection, IN/OUT would be switched. 
[...]
The design for years was to take the traffic from the "Destination Interface" perspective.

This is absolutely correct. Depending on perspective (external vs. internal), from a logical point of view, direction switches. And that's why taking the "destination interface" perspective just doesn't cut it. It leads to logic errors, as we have just established. 

Other firewall vendors solve this by letting the firewall know which networks are considered internal networks and which ones are considered external. On Checkpoint, for example, you explicitly define a network topology in the firewall config. You assign networks into different brackets (namely internal and DMZ, the rest it automatically considered external). Some other firewalls do this more simply by just using route lookups (anything that is not in the routing table, e.g. uses the default route, is considered external). Juniper uses a combination of zones and routes. 

Once it is determined whether the session initiator is external or internal, you can derive the correct "perspective", e.g. traffic direction.

If initiator is external, count like this:

Initiator sends data = INcoming
Initiator received data = OUTgoing

If initiator is internal, count like this:

Initiator sends data = OUTgoing
Initiator receives data = INcoming

Sophos XG already uses zones. It even has a designation for each zone. You could consider every zone designated with WAN role as external. Voila. 

Thanks for your suggestion.

What about other zones, which actually could be external (VPN Zone, MPLS Zones, Zones behind routing devices (For example Dark Fiber etc.))? You would consider them as external or Internal? Giving the administrator a option to decide about each zone could be a better approach. 

As said, there could be done many assumption on the firewall to find out, whether the traffic should be considered as IN or OUT. 

Do you know, how other vendor actually perform, if you create a ANY-ANY Rule? This will hit for internal and external Traffic, hence it will lead to weird traffic data counter, isnt it? Also rules, which implies LAN and external zone in it (LAN & WAN to LAN & WAN).  

Your examples makes perfectly sense, but are actually hard to archive and could lead to conflicts, which needs to be considered. 

PS: There are setups, which this will eventually break: XG without WAN Zone but Uplink to the Internet (eBGP, Upstream hosts etc.), Parent Proxy Setups. DNAT into a VPN Tunnel (Traffic coming from WAN, going into the VPN Tunnel. Is it Out, Out? Hard to tell). 

LuCar Toni 

the RX and TX are independent from the zone. RX and TX depends on the NIC where you are listening to.

For example:

your computer is sending a request to download data (it is not actually downloading the package) from an external public ip, what happens in terms of RX/TX?

your computer nic: high TX/low RX

nic of the switch where the computer is connected (ETH 0): high RX (because the port is receiving the request) / low TX

nic of the switch where the router is connected (ETH 23): high TX (because the same number of packets (with some modification) is sent / low RX

Now the package has been identified and the process starts again from the server high TX to router WAN port (high RX), high TX from router port to switch port ETH 23 (RX), ETH 0 TX and the computer nic RX.

This process applies in networking since 1960s.

Hope now it is clear.

It was clear in the beginning, but what are you suggesting? 

Are you see an issue in the way, XG is doing this? If you, please name it, maybe i can talk about this. 

RX/TX is just another phrase of IN/OUT (Interface/NIC Level). 

LuCar Toni said:

What about other zones, which actually could be external (VPN Zone, MPLS Zones, Zones behind routing devices (For example Dark Fiber etc.))? You would consider them as external or Internal? Giving the administrator a option to decide about each zone could be a better approach. 

That's actually what I was suggesting in my last post :)  By pointing out how other firewall vendors do it. They let admins define the network topology in some way, through which you designate external vs internal networks. 

LuCar Toni said:

Do you know, how other vendor actually perform, if you create a ANY-ANY Rule? This will hit for internal and external Traffic, hence it will lead to weird traffic data counter, isnt it? Also rules, which implies LAN and external zone in it (LAN & WAN to LAN & WAN).  

It has nothing to do with the rule, really. Whether the rule is any-any or more specific doesn't matter. The firewall still has to determine the incoming and outgoing interfaces of the connection. And it still has to determine who initiated the session in a flow. That's all it needs, when a topology (external vs internal) has been defined. 

LuCar Toni said:

Your examples makes perfectly sense, but are actually hard to archive and could lead to conflicts, which needs to be considered. 

It's not really hard to achieve, see other firewall vendors. It just has to be thought through and properly engineered. I am pretty sure Sophos engineers would be able to come up with a solution fairly easily. 

LuCar Toni said:

PS: There are setups, which this will eventually break: XG without WAN Zone but Uplink to the Internet (eBGP, Upstream hosts etc.), Parent Proxy Setups. DNAT into a VPN Tunnel (Traffic coming from WAN, going into the VPN Tunnel. Is it Out, Out? Hard to tell). 

Nope, nothing would break :)  See my comment above. All you need is a topology definition of some kind and then look at session establishment and which interfaces/zones the traffic traverses. And in complex, routed networks with BGP or MPLS and VPNs it's still all the same. You just have to define the topology from the perspective of the local firewall. It can be as simple as defining that all networks that the local firewall protects are considered internal networks and everything else is considered external (from that particular firewall's perspective).

Maybe you could DM/PM me some screenshots of other vendors, how they do it on their rule set? Just curious about their solution. 

Tried to find some information about this via research, but as other mentioned, other vendors are not showing such kind of traffic in their ruleset, as far as i could see. Could be wrong about that. 

No, I can't. It's the weekend. You just have to trust me on this. One example: Fortigate firewalls from Fortinet. You can enable a bytes column right in the ruleset. On Checkpoint, if you enable the Accounting feature, you right click on a rule, select "show in log", and it opens the log showing your bytes received, bytes sent, and total browse time (if it is app enabled) for every connection. It's similar on Palo Alto. 

Not sure why you doubt any of this.

On Checkpoint, if you enable the Accounting feature, you right click on a rule, select "show in log", and it opens the log showing your bytes received, bytes sent, and total browse time (if it is app enabled) for every connection.

It literally opens the own checkpoint logs for this, if you can do the same thing on XG for data counting, the only problem is, the XG logs is worse than checkpoint.

But I don't blame Sophos entirely for this, Their using a web based management solution, you can't compare to checkpoint smart console.

PAN and Checkpoint also uses Hit Counters by default instead of data counting, of course you can enable to shows Bytes A/B on it (On PAN.).

Also by default checkpoint uses Hit Counters on the rules instead of bandwidth as It is in XG.

Of course in the logs will show bandwidth,duration,sessions.But as stated before, don't compare smart console directly to XG.

Also, genuine question, Why data counting directly on the rule in XG is so important for you? What's the best use-case scenario for it?

Isn't it better to just use the Logs/Reporting function that already exists? I know both of them are bad compared to the competitors, but at least it "works".

Tried to find some information about this via research, but as other mentioned, other vendors are not showing such kind of traffic in their ruleset, as far as i could see. Could be wrong about that. 

They don't show it like XG.

Also after after 40 replies over this thread,Isn't it better to XG follow the industry standard and also use hit counters for the rules instead of just showing bandwidth, also please, show when the rule has first and last used.

Thanks!

Thanks for the log Prism.

Since RX/TX depend on the interface you are using, I suggest Sophos to maintain the counters an apply the counters to the first matching interface.

For LAN to WAN, TX is the traffic sent by the LAN and RX is the traffic received. Same thing from WAN to LAN: TX traffic sent by the WAN interface, RX traffic received. So in case of 10 MB file upload from WAN to LAN, RX is 10 MB, TX is few KBs.

Also I suggest Sophos to put back the reports per Inteface like UTM does.

Let's see what other users say about.

Please vote and consider this feature request:

https://ideas.sophos.com/forums/330219-xg-firewall/suggestions/34421893-report-of-traffic-for-each-wan-interface

Regards

Thanks for the log Prism.

Since RX/TX depend on the interface you are using, I suggest Sophos to maintain the counters an apply the counters to the first matching interface.

For LAN to WAN, TX is the traffic sent by the LAN and RX is the traffic received. Same thing from WAN to LAN: TX traffic sent by the WAN interface, RX traffic received. So in case of 10 MB file upload from WAN to LAN, RX is 10 MB, TX is few KBs.

Also I suggest Sophos to put back the reports per Inteface like UTM does.

Let's see what other users say about.

Please vote and consider this feature request:

https://ideas.sophos.com/forums/330219-xg-firewall/suggestions/34421893-report-of-traffic-for-each-wan-interface

Regards