more peering stuff, whats going on. - Page 2

JonathanLH · 19-02-2006, 20:28

zigzag is usually software rate limiters from my experience.
We have a 10mbit unmetered box in seattle, it's not set to 10mbit on the network card (god knows why), but is set from a cisco command on a gsr9000 series router. Due to this, a single thread download from say microsoft (also in seattle - with a 1ms ping!) runs at 10mbit on average, but zigzags nicely from about 8mbit to 12mbit.

Chrysalis · 19-02-2006, 22:30

I wonder if a level3 router somewhere is rate limiting then as this zig zag thing is happening on every transfer I do over level3 transit from the .us. But it is much more aggressive then what you mention, the transfer will be normal for about 30 secs which probably is enough to not affect small downloads and web browsing, then it will stall, start slowly rising and when hits a decent speed suddenly stall again and keep following this momentum. I am running no software rate limiters on either the server or my pc.

Ignition · 19-02-2006, 22:48

Quote:

Originally Posted by JonathanLH

zigzag is usually software rate limiters from my experience.
We have a 10mbit unmetered box in seattle, it's not set to 10mbit on the network card (god knows why), but is set from a cisco command on a gsr9000 series router. Due to this, a single thread download from say microsoft (also in seattle - with a 1ms ping!) runs at 10mbit on average, but zigzags nicely from about 8mbit to 12mbit.

If it were set at 10 on the network card you'd never get 10 due to the overheads. This way you average out a nice 10Mbit.

I doubt Level3 are throttling, I suspect more likely packet loss causing the stall, throttling won't make an outright stall but packet loss will.

Chrysalis · 20-02-2006, 15:00

I can get around 1.15mB/sec on 10 full duplex. My experience of isp's rate limiting isnt great I prefer good old fashion hardware limits if any limiting has to be done.

darkone338 · 23-02-2006, 21:43

i have a server at ServerMatrix.com (division of theplanet.com) and, despite it being on a 100mbit connection, and being able to transfer at close to 100mbit from other places, I get 4mbits MAX (on my 10mbit)... however interestingly... if I open another session, I can get another 4mbit (of course making my total download as 8mbit/sec)

Avg ping time, is 130ms to Dallas TX.

very strange

JonathanLH · 23-02-2006, 21:51

exactly the same here from our server at layeredtech, also in dallas, TX i believe

Chrysalis · 24-02-2006, 02:06

darkone that is normal, from .us you should expect 4-5 mbit per thread due to the rtt.

I was getting around 400kB at start for about 30seconds and it would rise to around 500kB slowly then suddenly drop to about 10kB and then slowly rise again to 300kB+ and drop, repeating.

JonathanLH · 24-02-2006, 08:26

i've had 10mbit before in 1 stream from the us, just depends on the bandwidth available.
although the rtt does have an impact, the tcp recieve window can offset that.
i knew a formula at one point that told you the max bandwidth for a given rtt and tcp window, but i forgot it

Chrysalis · 24-02-2006, 21:13

I may be mistaken here but you right it is possible but depends on the senders send window as well. I believe if you set a large recieve window which of course should be a multiple of mss, then if the sender has a small send window you dont get the full advantage of the large recieve window.

Also if that download was over http their was every chance it was coming down of a local proxy server.

Ignition · 26-02-2006, 18:33

Quote:

Originally Posted by JonathanLH

i've had 10mbit before in 1 stream from the us, just depends on the bandwidth available.
although the rtt does have an impact, the tcp recieve window can offset that.
i knew a formula at one point that told you the max bandwidth for a given rtt and tcp window, but i forgot it

It's easy really. Think about how much phatness you can handle at once, and how often you have to acknowledge data.

RWIN of 2000 for RTT of 100ms = max speed of 40,000 bytes / second if that helps (2000 bytes requiring an ack and can ack 20 times per second, each ack taking 1/2 of RTT)

So it's RWIN * ((1/RTT in ms) * 1000) * 2

Chrysalis · 27-02-2006, 00:37

some transits will cap transatlantic traffic I have seen it repeatedly.

cogent used to cap at 2mbit but now I think they cap at 5mbit.
level3 cap at 5mbit.
opentransit is at least 10mbit and is the best transatlantic performance I have seen.

telia and abovenet and others I dont know as I got no routes for servers going over that transit.

JonathanLH · 27-02-2006, 00:54

i've had more than 5mbit through cogent from my german servers to usa ones

Ignition · 27-02-2006, 10:25

Quote:

Originally Posted by Chrysalis

some transits will cap transatlantic traffic I have seen it repeatedly.

cogent used to cap at 2mbit but now I think they cap at 5mbit.
level3 cap at 5mbit.
opentransit is at least 10mbit and is the best transatlantic performance I have seen.

telia and abovenet and others I dont know as I got no routes for servers going over that transit.

That's completely untrue. Transit providers don't cap and don't care about bandwidth usage, more the better from their point of view.

EDIT:

http://www.dslreports.com/speedtests...net/1141036363

dslreports.com speed test result on 2006-02-27 05:32:43 EST:
7227 / 481
Your download speed : 7227 kbps or 903.4 KB/sec.

Your upload speed : 481 kbps or 60.1 KB/sec.

Tracing route to speedtest.cogeco.net [24.226.1.130]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms home.router [192.168.11.1]
2 * * * Request timed out.
3 8 ms 7 ms 7 ms gr1-stbc-rtr1.homechoice.co.uk [81.1.112.62]
4 9 ms 6 ms 7 ms 81-1-67-48.homechoice.co.uk [81.1.67.48]
5 8 ms 7 ms 7 ms 81-1-67-16.homechoice.co.uk [81.1.67.16]
6 7 ms 7 ms 7 ms 195.50.116.197
7 7 ms 7 ms 7 ms so-1-2-0.gar2.London1.Level3.net [212.113.0.118]
8 45 ms 7 ms 7 ms ae-0-56.bbr2.London1.Level3.net [4.68.116.162]
9 93 ms 93 ms 92 ms ae-0-0.bbr2.Chicago1.Level3.net [64.159.1.34]
10 96 ms 92 ms 93 ms ae-12-51.car2.Chicago1.Level3.net [4.68.101.3]
11 94 ms 93 ms 93 ms 4.79.64.14
12 104 ms 105 ms 105 ms rc2sh-pos13-0.mt.shawcable.net [66.163.77.13]
13 105 ms 105 ms 107 ms ra2sh-ge4-1.mt.shawcable.net [66.163.66.54]
14 105 ms 106 ms 106 ms rx0sh-cogeco-1.mt.bigpipeinc.com [66.244.255.2]
15 107 ms 107 ms 106 ms 24.226.0.102
16 107 ms 106 ms 109 ms cgowave-0-206.cgocable.net [24.226.0.206]
17 106 ms 106 ms 106 ms speedtest.cogeco.net [24.226.1.130]

Chrysalis · 27-02-2006, 17:46

it may not be them capping themselves then but I have never witnessed above 5mbit on a single thread from usa over the atlantic on some providers. So I have just gone on my own experience (not ntl). Their of course has been times where I have downloaded on ntl at 10meg speeds on what appears to be a us server but in that case it could be a eu/uk mirror, cached content on a proxy so I dont know where its coming from.

Regarding the comment from germany to usa I dont know I havent sent data that way before over cogent. Did you raise rwin on both the 2 servers sending and recieving?