FTP Sites - Authorization required

I would like to revive this issue, as it just bit me.

A vendor provider a download URL of the form ftp: // username : password @ hostname/path/filename.  (Spaces added to prevent this from becoming a hyperlink.)

Both Standard Mode and Transparent Mode proxies seem to strip the credentials.

• Using Standard Mode Web, the connection fails with an "Authentication Required" error messages.
• Using Transparent Mode FTP, times out or downloads a zero length file.  At least one time, Chrome prompted for basic-mode credentials, but the download still failed.

Log files were nearly useless, because web filter reported everything as passed, until I noticed that the logged URL had the credentials were stripped.

Eventually, I was able to download the file using an FTP client.

I have some other downloads that are failing, and I am having a hard time understanding why, because they are managed by product-specific software update managers, and my logs (web, ftp, firewall) have not been unhelpful.

Would it have been possible to make this work in a browser?

Does anyone have any advice for detecting when a problem of this sort is occurring?

If the browser is using the proxy explicitly, it handles FTP and the FTP Proxy doesn't get a look at the request.  If Web Filtering is in Transparent and FTP Proxy is in Both or Transparent, the FTP Proxy handles the request.  Could you tell from the FTP log whether it had handled the browser traffic at all?

I prefer to use Both and do FTP with FileZilla.

Cheers - Bob

I understood the theory, but the debugging process was challenging.

The Standard Web proxy, which also handles FTP, logged the traffic as passed, because it was not blocking anything.   But the remote site rejected the traffic because the credentials had been stripped.   I was looking for a block entry that was not there.

When I disabled Standard Web, Transparent Web and Transparent FTP automatically kicked in, but the logging for Transparent FTP is nearly useless (I do have "Log FTP data" enabled on the Firewall Advanced settings.)   Here is an excerpt:

2018:03:08-13:37:12 defense frox[24698]: Connect from 192.168.2.68
2018:03:08-13:37:12 defense frox[24698]: ... to 194.116.180.152()
2018:03:08-13:37:14 defense frox[24698]: PASV: 194.116.180.152:50048
2018:03:08-13:37:14 defense frox[24698]: Connection refused when trying to connect to 194.116.180.152
2018:03:08-13:37:14 defense frox[24698]: Failed to contact server data port
2018:03:08-13:40:07 defense frox[24698]: Client closed connection
2018:03:08-13:40:07 defense frox[24698]: Closing session

Eventually, I used an off-network laptop to prove that their server was working and the URL-embedded credentials were valid, so the problem HAD to be caused by UTM, even though UTM said it was letting everything through.

Just to add salt to the wound, when I tried to switch to an FTP client, I parsed the URL entry incorrectly so my login did not work at first.  Eventually, I got my credentials straight and finally understood that the Standard Web log was lacking credentials from the URL entry, so the credentials had been stripped.

It was a surprise that the credentials were not relayed, so I thought it was worth noting for the future.   

FTP with credentials is an odd configuration, since FTP is unencrypted for both login and data transfer.   So in a perfect world, this should not matter because no one should be using it.

Doug, don't you think you found a bug?  I don't think the Web Proxy should have stripped the credentials.

Cheers - Bob

Maybe so.  I will ask support if it is by design or by accident.   I tend to assume a lot of UTM's odd behaviors are considered features.

Hi Douglas, 

If this issue is already reported and investigated by the support team then please PM me the case ID so that it helps me to coordinate with them and the developers.

Thanks

Hi Douglas, 

If this issue is already reported and investigated by the support team then please PM me the case ID so that it helps me to coordinate with them and the developers.

Thanks