[Merged] FTP uploads intermittently fail

[Hodders]

In the last 3 or 4 weeks I've started to get strangeness with FTP transfers to a variety of sites (not NTL servers).

The transfer works fine for a random period (shortest 5 minutes to longest all day) and then I start getting connection timed out and lose the ability to FTP from that machine.

This happens from three seperate machines (all behind a Smoothwall doing NAT). Machines are running Win2k, WinXP and Linux.

After about 30 minutes the problem goes away until it happens again.

Rebooting the smoothwall router and/or the client in question makes no difference.

Has anyone else been getting issues of this type with NTL cable broadband ?

Someone in another forum mentioned that setting a lower MTU on the smoothwall outgoing interface might help ? Any opinions on this ?

[Hodders]

I've now been able to run a tcpdump on both the server and the client.

I can see the packets leaving my firewall but never arriving at the server until about 2 minutes later! The conclusion is that the FTP packets are being held up somewhere on the way. Given that access via ZEN adsl is fine, I can only conclude that ntlworld are delaying these packets somewhere on the way.

Running the login on a different client shows the packets working fine, ariving at the server and then being sent back and arriving at the client. So it seems to be sensitive to the NATed client ie the return port address.

Please help with any ideas !

[Ignition]

None to offer, ntl don't delay / throttle any ports, only process that happens is complete blocking of known worm ports as detailed earlier.

Where in the country are you? When did this issue start?

[tameruk]

Yep - same problem here.

In belfast, 700kbps connection, Linksys W54G router, BladeFTP client.

Trying to upload a 150meg file, just stops transfer after a few percent...

What's going on here do you think?

Running the login on a different client shows the packets working fine, ariving at the server and then being sent back and arriving at the client. So it seems to be sensitive to the NATed client ie the return port address.

Not exactly sure what you mean here, do you mean when you login via your zen adsl account?

Might be worth trying to switch to "passive" mode ftp transfer - will give it a go myself this weekend. AFAIK passive just means it doesn't rely on return packets for confirmation of data transfer at the client side. Useful for when you're ftp'ing from behind a firewall.

[rdhw]

Originally Posted by Hodders

I've now been able to run a tcpdump on both the server and the client.

I can see the packets leaving my firewall but never arriving at the server until about 2 minutes later! The conclusion is that the FTP packets are being held up somewhere on the way.

tcpdump only shows the user data that is sent or received: it does not show the protocol traffic on the wire, and the TCP recoveries and retransmissions in the event of failure.

A possible explanation of your observed "delay" is that there was no delay in NTL at all, but that a TCP packet was lost on the way from your FTP client to the remote server, or the TCP ACKs from the far end never made it back to your FTP client, so your FTP client sat there timing out and retransmitting the TCP packet until eventually it got through. An ethernet sniffer on the wire between your NAT router and the cable modem would reveal all.

A possible cause of packet loss might lie within your own network if, for instance, you have an ethernet duplex mismatch on any link. The best duplex setting depends on what model of cable modem you have.

Other causes are known to be Pace digital TV set top boxes (if that is what your cable modem is), if they are pushed anywhere near the upstream rate cap (as FTP uploads tend to do). You might get better upstream throughput by rate-limiting your aggregate upstream demand to be about 80% to 85% of the nominal upstream rate cap for your cable account. Then the STB will never get stressed anywhere near the upstream rate cap.

[altis]

Yey, Robin. Welcome back!

[Hodders]

I'll fill in some of the blanks...

1) The modem is a Surfboard 3100 ethernet modem going into a smoothwall based firewall.

2) Currently using passive FTP. May try using active FTP to see if that helps (I'll open up the ports from just the server in question).

3) Problem has only occurred in the last few (5 maybe) weeks.

4) Pinging the server always works with good <35ms delays.

5) I accept the point made about tcpdump.

6) The zen adsl test is done by setting the gateway address to the other end of a VPN to a different office, causing the traffic to go via a ipsec tunnel between the two. From our Bristol office that traffic then goes off to our colocated webserver.

The bottom line is that strangeness is going on. Most strange is that when one client stops working for 30mins or so the opther will all still work, until they fail.

It sounds like some router somewhwre is throwing a wobbly and waiting to time out and reset itself.......

Ho hum... Lets see if active ftp makes any diference.

[Neil]

Originally Posted by Hodders

I'll fill in some of the blanks...

1) The modem is a Surfboard 3100 ethernet modem going into a smoothwall based firewall.

2) Currently using passive FTP. May try using active FTP to see if that helps (I'll open up the ports from just the server in question).

3) Problem has only occurred in the last few (5 maybe) weeks.

4) Pinging the server always works with good <35ms delays.

5) I accept the point made about tcpdump.

6) The zen adsl test is done by setting the gateway address to the other end of a VPN to a different office, causing the traffic to go via a ipsec tunnel between the two. From our Bristol office that traffic then goes off to our colocated webserver.

The bottom line is that strangeness is going on. Most strange is that when one client stops working for 30mins or so the opther will all still work, until they fail.

It sounds like some router somewhwre is throwing a wobbly and waiting to time out and reset itself.......

Ho hum... Lets see if active ftp makes any diference.

Does the problem occur without the router/smoothwall box in the equation?

You really need to eliminate those 1st.

[rdhw]

Originally Posted by Hodders

The modem is a Surfboard 3100 ethernet modem going into a smoothwall based firewall.

In which case, you should manually configure the smoothwall WAN interface to be fixed at 10Mbps half-duplex, as the 3100 is incapable of supporting auto-negotiation correctly.

[Hodders]

I will try the half-duplex/10 MBps idea...

An excellent set of replies - thanks guys.....

I will also report back when/if I get anywhere as that way the thread will be usefull when people search on it in the future.

<EDIT>ethtool reports that 10/half is already set for the eth1 smoothwall i/f.

Also if this was causing the problems it would happen for all FTP sites and all intensive traffic.

I can download things like XP-SP2 quite happily.

Back to the passive/active ftp idea </EDIT>

Originally Posted by Hodders

3) Problem has only occurred in the last few (5 maybe) weeks.

Seems to rule out the speed/duplex issue?

[Hodders]

Tried 10/100 stuff.

1) Set to auto (original setting) - problem still occurs
2) Set to 10/half - problem occurs

I'll try FTP set to active and see if that helps....

Although the fact that other clients work when one won't makes me think that it is a NAT issue. Either:

A) the smoothwall router (although why would it suddenly start doing this ?), plus I would think the problem would still occur when going to different FTP sites...

B) The NTL network is going funny..... Hmmmmm surely not ?

Is anyone else getting this and have thay found a solution as it is really starting to irritate me....

Just tried the passive/active thing - Problems still there...

I've also just tried changing the IP address of a non-working client (test out the NAT theory....) and it still doesnt work !

ARRRGGGHHHHHHH

[Hodders]

I had the idea of trying NTL's own ftp server as if there is any transparent ftp proxing that is causing a problem then it may not appy here....

Upload the same set of files 10 times and no problems what-so-ever...

So that would tend to prove that:

a) It is not a problem with my network/firewall/cable modem.

b) It is a problem with NTL

c) It is a problem with the server I'm trying to upload to.

"c)" is kind of eliminated by the fact that the remote host still responds fine to requests by other clients....

So is there anything I can do about "b)" ?

Can I bypass and FTP proy that NTL may be running or route my info via a different route ?

[rdhw]

Originally Posted by Hodders

Can I bypass any FTP proxy that NTL may be running or route my info via a different route?

NTL don't proxy FTP. If it's only one of your LAN PCs affected, then you need to study the setup of that one PC carefully. If it is a Windows system, run HijackThis on it to see if there are any unexpected components present.

[Florence]

I had a similar problem last night it took over an hour to get a couple of pictures uploaded. the problem dissapeared as it suddenly arrived without anything being altered. I put it down to perhaps a UBR having a wobbler.