Guest User!

You are not Sophos Staff.

Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 11 replies
  • Subscribers 3 subscribers
  • Views 14971 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Slow Inter-vlan routing

RichardHughes1

Hello!

I'm in the need of some assistance, hopefully somebody could shed some light. The problem in which I am just about to discuss is indeed a problem in which I had previously experienced (or similar to) a good while back, but unfortunately, it was that long back that I can't jog my memory on what I did to resolve the issue.

The issue in which I am currently having is with data transfer between VLAN's. I had my installation of Sophos UTM running as a VM on a server running XenServer. Everything was running perfectly, but for a while, the physical server was running with it's resources maxed out (which isn't great). I decided to install XenServer on a second box, installed a dual NIC and then migrated the Sophos UTM VM to the second box. After making some adjustments with the network interfaces, everything was once again up and running.

Two days later, I started to notice excessive lag when transferring, say, a 5GB file, from the Client VLAN to the Server VLAN. The data transfer struggled to get beyond 5mbps, mostly sticking to 200-300kbps. Previously, I'd have got around 100-200mbps, when transferring data between VLAN's. I'm not sure if this problem is a result of migrating the Sophos UTM installation over to a different box or if this is the result of performing some updates to the XenServer installation itself.

I've stressed myself out again, where I've spent the entire day trying to work this one out. I've checked the Firewall and Intrusion Prevention. Nothing is really standing out to me and I haven't changed any of the settings in either the Firewall or Intrusion Prevention since migration. Strangely though, when checking IPS logs, I did come across this -

Total attacks blocked: 2
Rule ID Rule Description Rule group Packets %
1 38246 SERVER-OTHER Flexera FlexNet Publisher stack buffer overflow attempt Malware 1 50.00
2 38247 SERVER-OTHER Flexera FlexNet Publisher stack buffer overflow attempt Malware 1 50.00

When looking at those entries further, this appears to be when I was trying to transfer a 7gig file from the Server VLAN to the Client VLAN. This file does not contain a virus, so I am curious to why this has been flagged up?

Hopefully somebody out there can shed some light. This issue is starting to stress me out a bit now so overall, the issue is becoming even more difficult to resolve!

Cheers,

Richard

Forgot to mention! If I force a server and a client to use the L3 switch as the default gateway, data transfer between VLAN's is once again restored to 100+ mbps. If I then use the Sophos UTM as the default gateway, I once again struggle to get beyond 5mbps when transferring data between VLAN's.



This thread was automatically locked due to age.
Parents
  • 0

    Hi Richard,

    Please check if you discover any Dropped or Error packets on the interfaces. Run ifconfig through SSH.

    Did you test the speed by disabling IPS? I suggest you to try it and post an output. I am also curious which is the firmware version of UTM, if you are using v9.4 is Sandstorm feature causing the delay?

    Awaiting response.

    Thanks

  • 0 in reply to sachingurung

    Unknown said:

    Hi Richard,

    Please check if you discover any Dropped or Error packets on the interfaces. Run ifconfig through SSH.

    Did you test the speed by disabling IPS? I suggest you to try it and post an output. I am also curious which is the firmware version of UTM, if you are using v9.4 is Sandstorm feature causing the delay?

    Awaiting response.

    Thanks

    Hi Sachin,

    Thank's for your prompt response, I highly appreciate your help! I've ran ifconfig against the interfaces (whilst transferring a 7GB file from Client VLAN to Server VLAN). No packets were dropped by the looks of things -

    Eth0 is the interface which I have set as a trunk. All VLANs (1000, 1001, 1002 and 1003) pass through this interface. Eth1 is the interface which is external facing and Eth2 is the management interface.

    I was running version 9.4, but I reverted to a backup in which I had taken prior to migration, where the installation was still running 9.3, to see if the symptoms were still present. Unfortunately, the symptoms are still present after reverting back to 9.3. I'm not sure if something has gone wrong during the migration, could it be an issue with the interfaces, is it still looking for the hardware from the previous server? I did notice when using the backup which I had taken from the original server, that when I booted up the Sophos UTM on the second server, that I got the following message in the console -

    Could this be looking for non-existing interfaces? After a reboot, the above messages do not re-appear.

    When I migrated the Sophos UTM VM from the original XenServer, to the second, I exported the VM and then imported into the second XenServer. I then matched up the interfaces, so that they matched on the original VM. The MAC addresses for the interfaces have also not changed, I transferred these over so that they match the original installation.

    In regards to IPS, I have indeed disabled this as a test, but unfortunately, the transfer speed has not increased. IPS/Firewall hasn't been changed since I exported the installation from the original server, so I am not sure if this issue is down to the configuration of the UTM and possibly likely to be something with the interfaces after migration.

    Thanks again for your help!

    Richard

  • 0 in reply to RichardHughes1

    What happens if you set the MTU on the VLAN interfaces to something like 1200? (See #7 in Rulz.)

    Cheers - Bob

Reply Children
  • 0 in reply to BAlfson

    BAlfson said:

    What happens if you set the MTU on the VLAN interfaces to something like 1200? (See #7 in Rulz.)

    Cheers - Bob

    Hi Bob,

    Just gave this a try, changed it at both XenServer level and in the UTM itself. Unfortunately, the transfer speeds haven't increased. I'm actually considering building a second VM with a fresh installation of Sophos UTM and give it a basic config to see if I get similar or different results. Ran out of ideas, it's my birthday today and I've stressed my self out enough over this haha!

    Any other ideas?

    Cheers,

    Richard

  • 0 in reply to RichardHughes1

    Happy Birthday, Richard!  Often, I find my best way to solve a problem is to completely forget about it for awhile. [;)]

    Yeah, I was hoping you could avoid using tcpdump or WireShark...

    Cheers - Bob

  • 0 in reply to BAlfson

    BAlfson said:
    Happy Birthday, Richard!  Often, I find my best way to solve a problem is to completely forget about it for awhile. [;)]

    Yeah, I was hoping you could avoid using tcpdump or WireShark...

    Cheers - Bob

    Thank you very much Bob! I do agree, it does help to take a break from problems like these!

    Worryingly, I created a second VM on the second XenServer, where I performed a fresh installation of Sophos UTM 9.3. I then gave it a simple config with the VLANs applied. I then forced a mchine on the Client and Server VLANs to use the default gateway of the fresh UTM install. I then performed a copy and paste of a 7GB file from one machine to the other... 100-200kbps! I think from this, the UTM itself has probably been ruled out. I think this is starting to look like a hardware related issue or an issue with Sophos UTM running on the latest XenServer. I may try to do a fresh install of XenServer, without updates applied and then reimport the Sophos UTM VM.

    Alternatively, I could bypass the hyper visor completely and run the UTM straight from the box... How easy would you think it would be to migrate the virtual install, to a physical install? I'd assume that this would cause an issue NIC wise, where the interfaces will obviously change... Is there any commands which could be used to get me access to the management interface again?

    Cheers!

    Richard

  • 0 in reply to RichardHughes1

    Hi Richard,

    Happy Birthday :) 

    If IPS is disabled and there are no drops captured the next step is to reduce the MTU-MSS size as Bob suggested. If that did not help, I fear that the issue is not within Sophos UTM.

    Waiting until you complete your next exercise.

    Thanks

  • 0 in reply to RichardHughes1

    To change the order of NICs, as root on the console:

    # edit /etc/udev/rules.d/70-persistent-net.rules

    Save the file and restart the ASG so the new order is loaded.

    Cheers - Bob

    PS This is just one of many tricks listed in Goldy's great thread Astaro useful shell commands.

  • 0 in reply to BAlfson

    Well... I have some good news!

    I removed the drive with the XenServer installation to preserve that and then I attached another drive to perform a physical installation of the Sophos UTM (physically installed on the box, completely excluding XenServer). After the installation had completed, I restored the backup in which I had taken from the virtual installation, prior to shutting it down. Everything from that point, went pretty well! Transfer speeds between VLANs has rocketed back-up to 100-200mbps. Even though this is a gig network, I much prefer those speeds over 200 odd kbps!

    Any ideas on how I could push it towards 200+ meg? I realize that this is probably down to the read/write speeds of the target drive, so maybe I won't get above that.

    Cheers!

    Richard

  • 0 in reply to RichardHughes1

    Hi Richard,

    Wonderful, so I guess the issue was related to XenServer? To change the speed negotiation, navigate through Interfaces & Routing> Interfaces> Hardware> Edit> dis-select Auto negotiation,  here you will get an option to change the link mode. 

    Thanks

  • 0 in reply to sachingurung

    Hi again,

    Thanks for all your help, Sachin and Bob! In regards to the speed, the link speed is at 1 gig but what I was looking to do was to try and increase the data transfer rate. I have only ever managed to get to around 200mbps when transferring data across the network. Not sure if there is a way of increasing the frame size on the UTM? Not sure if that will make any difference to be honest.

    Cheers again!

    Richard