General : Can anyone explain...

[Eeeps]

... why I need 6+ threads (connections) to max out the downstream on my 150Mb/s broadband service but only one thread (connection) to max out a 1Gb/s ftp download from my NAS box?

I typically get about 20-30Mb/s on an ftp download to external sites using a single thread.

Is this down to inconsistent delivery times of individual TCP packets or out of order packets that I don't get on my internal network?

[Kymmy]

Totally different technologies. You can't really compare the differing media, distances, freqs

[MUD_Wizard]

It will depend on the O/S you're using?

Windows 7 and upwards has TCP Window scaling. Google it.

Congestion on your local VM connection is also a factor.

If your repeated tests via http://www.thinkbroadband.com/speedtest.html never get a green line similar to the yellow line then you may have local congestion on your DOCSIS channels. Or it could be congestion over your wifi.

My connection over wifi:

[img]Download Failed (1)[/img]


This will also give you a clue about factors when connecting to web servers at a distance: http://bradhedlund.com/2008/12/19/ho...istance-links/

The computer and cable modem will need to ramp up the throughput by negotiation with the end server.

Most if not all servers are rate limited. That is they will only deliver a limited speed per connection to the server. So multiple connections are required in order to get more throughput.

[Eeeps]

Originally Posted by Kymmy

Totally different technologies. You can't really compare the differing media, distances, freqs

Well, that's a very high level reason for the difference and the sort of answer a politician or 'IT' manager might give. But what's the real technical explanation?

---------- Post added at 21:53 ---------- Previous post was at 21:19 ----------

Originally Posted by MUD_Wizard

It will depend on the O/S you're using?

Windows 7 and upwards has TCP Window scaling. Google it.

Congestion on your local VM connection is also a factor.

If your repeated tests via http://www.thinkbroadband.com/speedtest.html never get a green line similar to the yellow line then you may have local congestion on your DOCSIS channels. Or it could be congestion over your wifi.

My connection over wifi:




This will also give you a clue about factors when connecting to web servers at a distance: http://bradhedlund.com/2008/12/19/ho...istance-links/

The computer and cable modem will need to ramp up the throughput by negotiation with the end server.

Most if not all servers are rate limited. That is they will only deliver a limited speed per connection to the server. So multiple connections are required in order to get more throughput.

Thanks for the reply,

Well, I've used Win 7, 8.1 and a Debian OS, all with similar results.

Your comment about congestion is an interesting one. I've used the Thinkbroadband speed tests and at 6am get a graph that's very similar to yours. After that my '1x' speed is circa 30Mb/s when the '6x' stays at 150Mb/s - why (back to my original question)?

I get a very good Thinkbroadband ping graph; at all times; even when I can only get 20Mb/s download on ftp. For me that's very interesting.

Seems to me that VM significantly prioritise IP packets based on historical rates to particular TCP sockets.

[Skie]

There are a lot of reasons. One of the most obvious ones if it's a consistent thing with the same download location would be throttling.

Some servers will throttle downloads (well, uploads from their perspective) to ensure that they have enough capacity available for a lot of users. Using multiple threads gets around this as long as they aren't being too clever about it and restricting by IP.

[MUD_Wizard]

Originally Posted by Eeeps

Your comment about congestion is an interesting one. I've used the Thinkbroadband speed tests and at 6am get a graph that's very similar to yours. After that my '1x' speed is circa 30Mb/s when the '6x' stays at 150Mb/s - why (back to my original question)?

Usually a combination of dropped packets (packet loss somewhere on the device-to-server link) and/or latency:
https://www.reddit.com/r/networking/...s_single_ones/

https://networkengineering.stackexch...ltiple-streams

http://blog.thinkbroadband.com/2015/...ensitive-test/

The packet loss may be due to a little congestion or other factors.

Originally Posted by Eeeps

I get a very good Thinkbroadband ping graph; at all times; even when I can only get 20Mb/s download on ftp. For me that's very interesting.

The BQM will be fine at low levels of congestion, where people can almost max out their connection using multiple threads.

---------- Post added at 05:19 ---------- Previous post was at 04:58 ----------

Originally Posted by Eeeps

Seems to me that VM significantly prioritise IP packets based on historical rates to particular TCP sockets.

Not sure what you mean in the context of downloads?

Try this FTP speed test: ftp://speedtest.tele2.net/

I was using the 1000GB file and could max out my connection now. Lets see what it's like during the day/evening.

[MUD_Wizard]

Originally Posted by MUD_Wizard

Try this FTP speed test: ftp://speedtest.tele2.net/

I was using the 1000GB file and could max out my connection now. Lets see what it's like during the day/evening.

Just ran this test again during peak time.

Still maxing out my connection at over 70Mb/s consistently (my tier speed) when downloading via FTP, producing network stats exactly as per this speed test:

[img]Download Failed (1)[/img]

After protocol overheads looks like 8.1-8.5MB/s, which equates to 64-68Mb/s.

So if FTP is treated differently I'm not seeing it. Maybe I need a faster connection. Though when I was on 152Mb tier I didn't have any problems either.

[ianch99]

Originally Posted by MUD_Wizard

Just ran this test again during peak time.

Still maxing out my connection at over 70Mb/s consistently (my tier speed) when downloading via FTP, producing network stats exactly as per this speed test:

[img]Download Failed (1)[/img]

After protocol overheads looks like 8.1-8.5MB/s, which equates to 64-68Mb/s.

So if FTP is treated differently I'm not seeing it. Maybe I need a faster connection. Though when I was on 152Mb tier I didn't have any problems either.

I can max out on 200Mbps from here: ftp://mirror.sov.uk.goscomb.net/debi...amd64/iso-dvd/

2016-02-28 11.33.18 pm.png

The tele2 site was a lot slower ..

[Eeeps]

Originally Posted by MUD_Wizard

Just ran this test again during peak time.

Still maxing out my connection at over 70Mb/s consistently (my tier speed) when downloading via FTP, producing network stats exactly as per this speed test:



After protocol overheads looks like 8.1-8.5MB/s, which equates to 64-68Mb/s.

So if FTP is treated differently I'm not seeing it. Maybe I need a faster connection. Though when I was on 152Mb tier I didn't have any problems either.

Wow. I'd love to get a graph like that, even on my 150Mb/s service...

[img]Download Failed (1)[/img]

But in the early morning it is like yours.
FTP at the moment is at about 10Mb/s

However, even during a speed test my ping stays at 8-15ms which to me suggests that 'bandwidth' is available but just not to the stream of packets that represent to the FTP socket.
It would be interesting to know the packet loss associated with a particular TCP socket rather than measured with ping. Is this an available stat on any stack?

Cheers,

Ian

[MUD_Wizard]

Originally Posted by Eeeps

Wow. I'd love to get a graph like that, even on my 150Mb/s service...

I had a nice flat line for both green and yellow around 160Mb when I was on the 152Mb tier.

Originally Posted by Eeeps

But in the early morning it is like yours.

Yours looks fairly well congested, somewhere along the line.

Originally Posted by Eeeps

FTP at the moment is at about 10Mb/s

See your green line, that's what you're getting.

Originally Posted by Eeeps

However, even during a speed test my ping stays at 8-15ms which to me suggests that 'bandwidth' is available but just not to the stream of packets that represent to the FTP socket.

No, it just means congestion hasn't gotten bad enough to start dropping or delaying lots of ping packets.

Originally Posted by Eeeps

It would be interesting to know the packet loss associated with a particular TCP socket rather than measured with ping. Is this an available stat on any stack?

On Windows use Windows Resource Monitor, network tab.

Though you'll probably see 0% packet loss.