v18: Bug with data counting in firewall rules?

Hi cryptochrome, 

Could you please share the screenshot of the firewall rule that shows data in and out counters? I will verify if it should be like that or not and update you.

Thanks,

Hi cryptochrome, 

Could you please share the screenshot of the firewall rule that shows data in and out counters? I will verify if it should be like that or not and update you.

Thanks,

Sure, here you go:

Firewall Rule:

Corresponding NAT rule:

Firewall Rule Details:

Note that this is just one example. I am seeing the same "reversed" counter on other incoming rules. 

How do you count the data? If someone on the internet initiates the connection and transfers a lot of data, does that count as incoming or outgoing?

LuCar Toni said:

V17.5 Client to Server through DNAT.

Same behavior like V18. 

I am talking about v18, not v17. If v17 has done it like this that's fine, I never noticed and hence didn't care. Now with v18 I noticed. 

LuCar Toni said:

 

Actually that is not true.

LAN to WAN. Download shown as IN Bytes.

DMZ to WAN. Download shown as IN Bytes.

WAN to DMZ(DNAT). Download shown as IN Bytes. 

You're still not grasping the concept of session initiation. It doesn't matter which interface combination (LAN to WAN or the other way around). The only thing that matters is who initiated the connection. You can't seem to wrap your head around it because you keep thinking about this from the client perspective you are used to. But it's wrong. Try to think outside the box. In your WAN to DMZ (DNAT) example above, the client (WAN) is downloading from a server in the DMZ, I assume. The server is sending data TO the client, OUT the WAN interface through the DMZ interface. The only thing that the traffic direction must be derived from is the session initiator. You send data TO the initiator, you send data OUT. You receive data FROM the initiator, data is coming IN. It's really as simple as that. 

If you just want to be right and think the way Sophos XG is doing it now, that's fine. It's still illogical and inconsistent, because the behavior is different between SNAT and DNAT (and yes, that is actually true, at least that's what I see here on v18). 

Updated my post above.  

Client is always the session initiator. Hence we are using In for traffic going to the Session initiator in all perspectives, no matter if NAT or not. 

Proved it via Screenshots. 

XG act like it should act, always collecting the session initiator and summarize the traffic as IN to the initiator, and OUT coming from the initiator. 

I assume there is nothing wrong with this way to deal with it. 

LuCar Toni, 

can you investigate on the issue inside?

You haven't proven anything other than not understanding, sorry. Client is NOT always the session initiator. You're factually wrong. And just because you always did it like this doesn't make it right. There are two other people in this thread who are as confused about this as I am.

And yes, you are actually doing it differently when DNAT is applied. Here is my "prove". This is a firewall rule for access to the internet, where users predominantly download stuff from the internet (hence, they are the session initiators). The rule is subject to SNAT(MASQ). It correctly shows most data as incoming and only little data as outgoing, as expected:

This is a firewall rule that has a corresponding DNAT rule. It shows that the server in the DMZ is sending very little data and it looks like the server is downloading a lot (IN), but it's actually the other way around, the server is SENDING a lot of data to people on the internet (who are the session initiators). The numbers are reversed:

So there you have it, it is clearly and obviously different when DNAT is in place.

Don't assume you are right. Consider you might be wrong. Other people in this thread agree.

I cannot give more input than above.

My Points are simply:

V17.5 to V18 - No Changes in Firewall Rule handling regarding data count.

In / Out will be tracked based on the Session Initiator (https://tools.ietf.org/html/rfc675).

In is traffic going to the initiator, Out is traffic going to the session recipient. 

It is a Term problem. Plex is the server, Clients are the initiator. You do not understand my point, which i try to explain. XG is showing you in Rule 9, the clients are getting 1.14 GB data. Clients are the session initiator, the Clients in the Internet. 

Sorry  - I cannot do more than that. 

Thanks Lucar.

LuCar Toni said:

I cannot give more input than above.

My Points are simply:

V17.5 to V18 - No Changes in Firewall Rule handling regarding data count.

In / Out will be tracked based on the Session Initiator (https://tools.ietf.org/html/rfc675).

In is traffic going to the initiator, Out is traffic going to the session recipient. 

 

But that is NOT what you are doing. If you think that's what you are doing, then we are actually looking at a bug. Because in my case, the firewall is not reporting the actual and factual directionality of the traffic based on session initiator. It's doing the opposite. 

Also, nobody disputed that this behavoir has not changed from v17 to v18. 

Last try from my site. I am simply here to try to help to understand what is going on.

https://support.plex.tv/articles/201543147-what-network-ports-do-i-need-to-allow-through-my-firewall/

Your Mobile phone is building a connection to your XG Server, port 32400. 

XG look up the NAT and Firewall Table to find a proper NAT and Firewall Rule.

Assuming DNAT and the correct Firewall Rule (WAN to DMZ). 

XG will forward the Syn Packet to the plex server port 32400.

1.2.3.4:12345 -> WAN IP:32400 [S] 

1.2.3.4:12345 -> Plex IP:32400 [S]

Plex will respond with SYN/ACK.

Plex IP:32400 -> 1.2.3.4:12345 [S.]

WAN IP:32400 -> 1.2.3.4:12345 [S.]

ACK from Device for Three-way Handshake.

1.2.3.4:12345 -> WAN IP:32400 [.] 

1.2.3.4:12345 -> Plex IP:32400 [.]

Session established. 

1.2.3.4 is the Initiator. 

Plex IP is the Recipient. 

All Devices have this connection mapped. XG, Mobile and Server. 

Assuming the Client asking for a Movie, he pushes this little packet to the Server. 

Lets say, those packets are 1 MB big in general. 

1.2.3.4:12345 -> WAN IP:32400 [P. GIVE ME MOVIE] 

1.2.3.4:12345 -> Plex IP:32400 [P. GIVE ME MOVIE] 

Server is responding with the Movie. 

Lets say, this Movie is 20 GB.

Plex IP:32400 -> 1.2.3.4:12345 [P. MOVIE DATA BIG]

WAN IP:32400 -> 1.2.3.4:12345 [P. MOVIE DATA BIG]

Now XG is seeing this Traffic. 

As mentioned earlier, we are using:

IN for Traffic to the Initiator.

OUT for Traffic to the Recipient. 

The Firewall will now track:

IN: [P. MOVIE DATA BIG] & [S.]

OUT: [P. GIVE ME MOVIE]  & [S] & [.]

Lets assume we take your other Rule for granted.

LAN to WAN - ANY services. 

Same construct like above is used.

Your Client is trying to download a Windows Update.

Client IP:12345 -> Microsoft IP:443 [S]

WAN IP:12345 -> Microsoft IP:443 [S]

Microsoft IP:443 -> WAN IP:12345 [S.]

Microsoft IP:443 -> Client IP:12345 [S.]

Client IP:12345 -> Microsoft IP:443 [.]

WAN IP:12345 -> Microsoft IP:443 [.]

Handshake done. 

Client IP:12345 -> Microsoft IP:443 [P. Give me Update / Little Request]

Microsoft IP:443 -> WAN IP:12345 [P. Update / Big Packets]

Firewall Rule will now pick up:

IN [P. Update / Big Packets] & [S.]

OUT: [P. Give me Update / Little Request] & [S] & [S.]

As mentioned earlier, I completely understand how you do it, but I am saying it's wrong, because it does not reflect the actual traffic direction. Using IN when sending traffic TO the initiator is the problem. When you send someone out of your house, you are not saying you are letting him in, do you? Sophos XG does. 

Using IN when sending data TO someone is just utterly illogical. 

For what it's worth: I know basically every firewall vendor out there. Checkpoint, Fortinet, Palo Alto, Cisco, Juniper, you name it. They all do it the way I described. Only Sophos has to introduced their own weird concept that doesn't reflect reality. For reasons that are beyond me. 

I give up. You insist on not understanding, and you insist on "Sophos is right", so it's pretty much pointless to argue with you. I would have wished that Sophos (who you seem to represent here) would take customer feedback more serious instead of just saying: We are right, because we have always done it like this, we are not considering your opinion, deal with it. 

LuCar Toni said:

LAN to WAN:

Download Scenario: Client sends 5 Bytes to XG. Server sends 10 MB to XG.  XG Sends 5 Bytes to Server, XG Sends 10 MB to the Client. 

From XG Perspective: 

Firewall: IN 10 MB, OUT 5 Bytes. 

Firewall: Sent 10 MB to the Client, Sent 5 Bytes to the Server. 

Firewall: Gets 10 MB from the Server, Gets 5 Bytes from the Client.

Firewall: Downloads 10 MB from the Server, Uploads 5 Bytes to the Server.

Firewall: Downloads 5 Bytes from the Client, Uploads 10 MB to the Server. 

DNAT Scenario:

Download Scenario: WebClient sends 5 Bytes to XG. DMZ Server sends 10 MB to XG.  

Firewall: IN 10 MB, OUT, 5 Bytes. 

Firewall: Sent 10 MB to the WebClient, Sent 5 Bytes to the DMZ Server. 

Firewall: Gets 10 MB from the DMZ Server, Gets 5 Bytes from the WebClient.

Firewall: Downloads 10 MB from the DMZ Server, Uploads 5 Bytes to the  DMZ Server.

Firewall: Downloads 5 Bytes from the WebClient, Uploads 10 MB to the WebClient. 

Sorry Lucar, but I do not agree with you at all.

Traffic starts to count when the first point starts the connection. So if we are in a situation LAN to WAN, the Client sends a request on an external website to GET a package, so the server responds "OK this is the 10 MB package" and bla bla bla, the download starts. So on the WAN interface, I expect to see Inbound 10 MB while Outbound just few KBs.

The same applies for WAN to LAN. I expect to see that outbound of WAN port is 10 MB while the inbound are just few KBs.

This is the universal approach.

From advanced shell use the "old" iftop command in this way:

iftop -i Port2

once inside use the "t" letter to switch between TX and RX and at the same time start to download something from internet.

Same thing for traffic from WAN to LAN.

Please update with the test. I do not want to connect to a customer's XG box today for a test as it is Saturday.

LuCar Toni said:

LAN to WAN:

Download Scenario: Client sends 5 Bytes to XG. Server sends 10 MB to XG.  XG Sends 5 Bytes to Server, XG Sends 10 MB to the Client. 

From XG Perspective: 

Firewall: IN 10 MB, OUT 5 Bytes. 

Firewall: Sent 10 MB to the Client, Sent 5 Bytes to the Server. 

Firewall: Gets 10 MB from the Server, Gets 5 Bytes from the Client.

Firewall: Downloads 10 MB from the Server, Uploads 5 Bytes to the Server.

Firewall: Downloads 5 Bytes from the Client, Uploads 10 MB to the Server. 

DNAT Scenario:

Download Scenario: WebClient sends 5 Bytes to XG. DMZ Server sends 10 MB to XG.  

Firewall: IN 10 MB, OUT, 5 Bytes. 

Firewall: Sent 10 MB to the WebClient, Sent 5 Bytes to the DMZ Server. 

Firewall: Gets 10 MB from the DMZ Server, Gets 5 Bytes from the WebClient.

Firewall: Downloads 10 MB from the DMZ Server, Uploads 5 Bytes to the  DMZ Server.

Firewall: Downloads 5 Bytes from the WebClient, Uploads 10 MB to the WebClient. 

Sorry Lucar, but I do not agree with you at all.

Traffic starts to count when the first point starts the connection. So if we are in a situation LAN to WAN, the Client sends a request on an external website to GET a package, so the server responds "OK this is the 10 MB package" and bla bla bla, the download starts. So on the WAN interface, I expect to see Inbound 10 MB while Outbound just few KBs.

The same applies for WAN to LAN. I expect to see that outbound of WAN port is 10 MB while the inbound are just few KBs.

This is the universal approach.

From advanced shell use the "old" iftop command in this way:

iftop -i Port2

once inside use the "t" letter to switch between TX and RX and at the same time start to download something from internet.

Same thing for traffic from WAN to LAN.

Please update with the test. I do not want to connect to a customer's XG box today for a test as it is Saturday.

This is how XG is doing it on all installations. 

I will not update the iftops for now, they will simply reflect the same thing. 

Luk, you need to keep in Mind, Firewall Logging will be the same, if you talking about WAN, LAN,DMZ etc. XG will not start to act differently, because you have a DNAT. 

So do not worry, everything is the same like on V17.5. I could not find any inconsistency in the logging but a discussion about whether it should be called IN for Value A or called OUT for Value A. The Values are correct. 

As told earlier, i am not saying this is the best solution for everything, it is more likely a "term issue". 

Also checked ideas.sophos.com. Could not find a feature request to reverse those values. 

My suggestion would be to have this parameters variable and let the customer choose, how XG should show the traffic on this particular firewall rule. 

PS: If Sophos change those values, they would not make any sense to me.

LAN to WAN with Big OUT and Low IN, looks odd to me. 

WAN to DMZ would be fine with those switch. BIG OUT, Low IN. 

Sophos could simply remove IN/OUT Terms to meet other vendors. 

Sometimes you do not have to agree with everybody, cryptochrome . That is actually my personal opinion like always. I am posting as a person here, not a Sophos Employee in my Free Time.

After setting all cards to the table, other should revisit the post and decide, if they agree or not. 

Prism FloSupport NateP2 

After your first post, it looked like there is a Bug, which is not. 

So you are agree with me, there is no Bug nor inconsistency in the whole process? 

The only point, which you want to have, is to change/switch both values for A/all firewall rules, correct? 

If you would change it for all Rules, maybe other firewall rules could be not logically anymore. 

LuCar Toni said:

PS: If Sophos change those values, they would not make any sense to me.

Obviously.

And that's the whole problem here. You still don't understand the issue. And you refuse to even consider that something you don't understand ("make any sense to me") could actually be right.

I guess ignorance is bliss.

LuCar Toni said:

 

Sophos could simply remove IN/OUT Terms to meet other vendors. 

Since you edited your post while I was typing my response, here is my reply to the additions you made to it:

LuCar Toni said:

 

So you are agree with me, there is no Bug nor inconsistency in the whole process? 

God, this is painful. No, I do not agree with you. Because you refuse to understand the issue. I have repeatedly stated that there is in fact an inconsistency, because the data counting actually works fine on outgoing, SNAT/MASQ traffic. It works differently on incoming DNAT traffic. So there is at least an inconsistency. Whether that inconsistency should be labeled as a bug is not for me to decide (although I would definitely see it as one).  

LuCar Toni said:

The only point, which you want to have, is to change/switch both values for A/all firewall rules, correct? 

If you would change it for all Rules, maybe other firewall rules could be not logically anymore. 

No :-/ Not correct. I do not want values changed for all firewall rules. Only for those where the indication of traffic direction is actually wrong (from what I can tell, that is only relevant to incoming DNAT traffic). 

But you're not listening, so whatever. 

LuCar Toni said:

So you are agree with me, there is no Bug nor inconsistency in the whole process? 

I understand what cryptochrome is trying to explain to you, but in my opinion, this isn't a bug.

I don't even care about the data counting in the rules, every other NGFW vendor I've used never showed the data counting as XG shows, they always showed in the Rule "Hit Counter". At the same time showing the first time the Rule has been used, and the last time It has been used. Which is much better than showing just the amount of data transferred over that rule in XG right now.

If you wanted to see the traffic bandwidth being transferred over a rule, you would use the Logs or the Reporting system of the appliance. The only problem of doing this in XG, is because the current logs & reporting system is bad compared to the competition.

Thanks!

Please moderate with the terms. If someone here can open a ticket with the support to investigate can really help.

For reporting, I do not like them compared to competition and to UTM 9 either.

I have the doubt that this in/out issue generates false reports too.

As feature, I am waiting reports per interface where you can start from interface and drill down to services, applications, users and so on. This is one of the most reporting I am using on UTM and on other products.

Thanks

Maybe i did a bad Job in explaining the underlying issue.

Will try it again, because i understand, why this is causing an issue to people, but to resolve this, it is not quite easy.

You have different perspectives for Firewall logging. You can perform it on different level.

Interface Level (LAN or WAN Interface). Connection based (Client to Server, how much Data is processed - No IN/OUT).  

Logging on the Interface, you have always the perspective IN/OUT. A Firewall always(usually) interact with two Interfaces in a basic communication. So you have two IN/OUT for the same Connection. You have the Destination Interface and the Source Interface. 

Logging on the Connection, you would have Data send by A and by B. There is no UP/DOWN, only Bytes from A / Bytes from B. Or Bytes from Client, Bytes from Server.   

Lets say, you are establish a connection between LAN Client and WAN Server. You could start to log the traffic now based on the Traffic from the WAN Interface or LAN interface. 

From the LAN Interface, the Connection could be Logged as IN/OUT.

From the WAN Interface, the same Connection, IN/OUT would be switched. 

From the Connection based, you would have Bytes by LAN Client and Bytes by WAN Server. 

Lets now look at XG. The design for years was to take the traffic from the "Destination Interface" perspective. 

That made sense for the use Case of LAN to WAN. From the WAN interface perspective, Servers are sending date (IN) and Clients are sending data (OUT). 

For DNAT, this could be an issue. The Destination Interface is DMZ. So the Client in WAN is sending Traffic (OUT), the Server in DMZ is Sending traffic (IN).

If you leave the LAN/WAN concept and look at LAN to DMZ, you would have a certain expectation: DMZ Interface as Logging Interface: Because most likely your Client (LAN) is talking to a Server (DMZ).

But this get more and more complicated if you take the bigger picture. You have LAN to LAN. What are you logging? DMZ to DMZ? (two Interfaces in the same Zone). You have your own created Zone? 

Should XG automatically figure out which Interface is used and switching the logging based on the connection Destination? Could be "misleading" as well. I am not able to figure out a solution but to give the administrator the possibility to choose at their own, which way around the logging should be done. 

What would be the solution? cryptochrome Maybe you could give us some insights, how other vendors do this? Because as far as i know, they most likely only log the Connection (How much traffic is proceed with this rule) without any IN/OUT indicator. That is my finding at the moment. Please consider the bigger picture in resolving this issue. 

*Edit* Forget one Use Case: Maybe use the WAN interface, then switch between IN/OUT because it is uploaded? But this would not work for VPN, MPLS etc. Also no solution. 

As Prism explained, most NGFW are only using the Connection logging for their Firewall rules (which would be only sent by A / B, not IN/OUT). 

Hope this is clear. If not, we can continue this Discussion via PM, if you want. Happy to explain this in more detail, why there is no inconsistency in the current Solution. 

PS: Reports are generated based on the connection level. 

Luca, thanks for explaining again in detail. I fully understand how it works, and I completely understand where you see the problem:

From the LAN Interface, the Connection could be Logged as IN/OUT.
From the WAN Interface, the same Connection, IN/OUT would be switched. 
[...]
The design for years was to take the traffic from the "Destination Interface" perspective.

This is absolutely correct. Depending on perspective (external vs. internal), from a logical point of view, direction switches. And that's why taking the "destination interface" perspective just doesn't cut it. It leads to logic errors, as we have just established. 

Other firewall vendors solve this by letting the firewall know which networks are considered internal networks and which ones are considered external. On Checkpoint, for example, you explicitly define a network topology in the firewall config. You assign networks into different brackets (namely internal and DMZ, the rest it automatically considered external). Some other firewalls do this more simply by just using route lookups (anything that is not in the routing table, e.g. uses the default route, is considered external). Juniper uses a combination of zones and routes. 

Once it is determined whether the session initiator is external or internal, you can derive the correct "perspective", e.g. traffic direction.

If initiator is external, count like this:

Initiator sends data = INcoming
Initiator received data = OUTgoing

If initiator is internal, count like this:

Initiator sends data = OUTgoing
Initiator receives data = INcoming

Sophos XG already uses zones. It even has a designation for each zone. You could consider every zone designated with WAN role as external. Voila. 

Thanks for your suggestion.

What about other zones, which actually could be external (VPN Zone, MPLS Zones, Zones behind routing devices (For example Dark Fiber etc.))? You would consider them as external or Internal? Giving the administrator a option to decide about each zone could be a better approach. 

As said, there could be done many assumption on the firewall to find out, whether the traffic should be considered as IN or OUT. 

Do you know, how other vendor actually perform, if you create a ANY-ANY Rule? This will hit for internal and external Traffic, hence it will lead to weird traffic data counter, isnt it? Also rules, which implies LAN and external zone in it (LAN & WAN to LAN & WAN).  

Your examples makes perfectly sense, but are actually hard to archive and could lead to conflicts, which needs to be considered. 

PS: There are setups, which this will eventually break: XG without WAN Zone but Uplink to the Internet (eBGP, Upstream hosts etc.), Parent Proxy Setups. DNAT into a VPN Tunnel (Traffic coming from WAN, going into the VPN Tunnel. Is it Out, Out? Hard to tell). 

LuCar Toni 

the RX and TX are independent from the zone. RX and TX depends on the NIC where you are listening to.

For example:

your computer is sending a request to download data (it is not actually downloading the package) from an external public ip, what happens in terms of RX/TX?

your computer nic: high TX/low RX

nic of the switch where the computer is connected (ETH 0): high RX (because the port is receiving the request) / low TX

nic of the switch where the router is connected (ETH 23): high TX (because the same number of packets (with some modification) is sent / low RX

Now the package has been identified and the process starts again from the server high TX to router WAN port (high RX), high TX from router port to switch port ETH 23 (RX), ETH 0 TX and the computer nic RX.

This process applies in networking since 1960s.

Hope now it is clear.

It was clear in the beginning, but what are you suggesting? 

Are you see an issue in the way, XG is doing this? If you, please name it, maybe i can talk about this. 

RX/TX is just another phrase of IN/OUT (Interface/NIC Level).