VM Routing and CloudFlare.com
 

Hi,

I signed up to CloudFlare.com a few days ago, and have noticed my trace routes go via the US and not via Amsterdam, NL, like they should.

So I contacted CloudFlare and they say: 'Virgin Media, for some reason, is preferring a US peering route to a European one'.

So I am wondering who I can get in contact with about this?
I have nothing against the off-shore or Customer service, but I think it could be over there heads.

Virgin Media connection
C:\Users\Owner>tracert dev.social-mates.co.uk

Tracing route to dev.social-mates.co.uk [199.27.135.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 58 ms 8 ms 9 ms 10.xxx.144.1
3 7 ms 8 ms 8 ms nott-core-1a-ge-310-2385.network.virginmedia.net [82.13.49.221]
4 10 ms 11 ms 9 ms leed-bb-1a-as0-0.network.virginmedia.net [213.105.174.169]
5 14 ms 17 ms 15 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253.185.238]
6 16 ms 14 ms 15 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.105.159.6]
7 14 ms 15 ms 15 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.184.6]
8 156 ms 88 ms 89 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.115.61]
9 97 ms 102 ms 123 ms as23352.xe-2-1-0-82.cr2.iad1.us.nlayer.net [69.31.30.102]
10 92 ms 89 ms 89 ms unassigned.midatlanticbb.com [66.225.125.1]
11 88 ms 89 ms 89 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]

So I went to traceroute.org and did 3 trace routes from 3 different sites in the UK, which are below.

If you noticed the below 3 all go via: ge0-3.aggrb3.ams3.nl.scnet.net
Which is the Netherlands.

If I do a trace route via http://www.xciv.org/cgi-bin/traceroute
Put in: dev.social-mates.co.uk
Trace route is:

traceroute to dev.social-mates.co.uk (199.27.135.74): 1-30 hops, 38 byte packets
1 ge-0-0-3-113.br1.ix1lon.uk.as34706.net (194.145.196.130) 2.54 ms (ttl=254!) 2.51 ms (ttl=254!) 2.74 ms (ttl=254!)
2 ge-0-0-0-90.br1.thelon.uk.as34706.net (194.145.196.227) 28.7 ms (ttl=253!) 29.3 ms (ttl=253!) 30.1 ms (ttl=253!)
3 po3-1363-cr0.tch.uk.as6908.net (78.41.155.50) 1.37 ms 1.57 ms 1.48 ms
4 te3-4-3501-cr0.nik.nl.as6908.net (78.41.154.46) 7.30 ms 7.44 ms 7.36 ms
5 xe-4-3.r01.amstnl02.nl.bb.gin.ntt.net (81.20.64.41) 8.06 ms 7.91 ms 7.63 ms
6 xe-3-0-1.ar1.ams3.nl.nlayer.net (69.22.139.204) 15.1 ms xe-3-0-3.ar1.ams3.nl.nlayer.net (69.22.139.202) 16.9 ms 15.4 ms
7 as23352.vlan-102.ar1.ams3.nl.nlayer.net (69.22.139.123) 15.2 ms 14.9 ms 14.7 ms
8 ge0-3.aggrB3.ams3.nl.scnet.net (205.234.220.229) 16.9 ms ge0-4.aggrB3.ams3.nl.scnet.net (205.234.220.231) 16.5 ms ge0-3.aggrB3.ams3.nl.scnet.net (205.234.220.229) 15.2 ms
9 cf-199-27-131-10.cloudflare.com (199.27.131.10) 14.4 ms (ttl=56!) 14.1 ms (ttl=56!) 14.2 ms (ttl=56!)
10 cf-199-27-135-74.cloudflare.com (199.27.135.74) 14.2 ms (ttl=56!) 14.4 ms (ttl=56!) 14.5 ms (ttl=56!)


If I do a trace route via: http://ab.newnet.co.uk/cgi-bin/traceroute
Put in: dev.social-mates.co.uk
Trace route is:

traceroute to dev.social-mates.co.uk (199.27.134.74): 1-30 hops, 38 byte packets
1 telh2.newnet.co.uk (212.87.64.1) 0.214 ms (ttl=64!) 0.203 ms (ttl=64!) 0.199 ms (ttl=64!)
2 7609-1.lan2.newnet.co.uk (212.87.79.65) 0.613 ms 0.528 ms 0.326 ms
3 ldn-s2-rou-1041.UK.eurorings.net (134.222.109.85) 0.525 ms (ttl=252!) 0.541 ms (ttl=252!) 0.450 ms (ttl=252!)
4 xe-0-3-0.cr1.lhr1.uk.nlayer.net (195.66.224.37) 1.14 ms 3.92 ms 1.07 ms
5 ae2-70g.cr1.lhr1.uk.nlayer.net (69.22.139.62) 0.648 ms (ttl=250!) 0.800 ms (ttl=250!) 0.700 ms (ttl=250!)
6 xe-4-0-0.cr1.ams2.nl.nlayer.net (69.22.142.95) 6.89 ms (ttl=247!) 7.17 ms (ttl=247!) 7.07 ms (ttl=247!)
7 as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net (69.22.139.119) 7.64 ms (ttl=247!) as23352.xe-4-2-0-105.cr1.ams2.nl.nlayer.net (69.22.139.121) 7.69 ms (ttl=247!) as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net (69.22.139.119) 7.55 ms (ttl=247!)
8 ge0-6.aggrB3.ams3.nl.scnet.net (205.234.220.235) 10.0 ms (ttl=246!) 52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 7.54 ms (ttl=247!) ge0-6.aggrB3.ams3.nl.scnet.net (205.234.220.235) 8.42 ms (ttl=246!)
9 cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.39 ms (ttl=54!) 7.42 ms (ttl=54!) ge0-5.aggrB3.ams3.nl.scnet.net (205.234.220.233) 7.82 ms (ttl=246!)
10 cf-199-27-134-74.cloudflare.com (199.27.134.74) 7.24 ms (ttl=54!) 7.44 ms (ttl=54!) cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.47 ms (ttl=54!)


If I do a trace route via: http://www.erik.co.uk/lg/
Put in: dev.social-mates.co.uk
Trace route is:

traceroute dev.social-mates.co.uk
(please be patient, this script may take a moment before you see anything)

1 46.37.32.129 (46.37.32.129) 1.279 ms 0.527 ms 5.571 ms
2 thn-gw01.cerberus.net.uk (46.37.32.1) 0.177 ms 0.169 ms 0.163 ms
3 ge-0-3-83.lon-th12gw.spn.kcom.com (62.164.215.9) 0.412 ms 0.409 ms 0.465 ms
4 134.222.146.53 (134.222.146.53) 0.477 ms 0.664 ms 0.465 ms
5 ldn-s10-rou-1041.uk.eurorings.net (134.222.146.52) 0.730 ms 0.767 ms 0.712 ms
6 ldn-s2-rou-1003.uk.eurorings.net (134.222.231.29) 0.730 ms 0.769 ms 0.714 ms
7 xe-0-3-0.cr1.lhr1.uk.nlayer.net (195.66.224.37) 1.288 ms 3.096 ms 0.754 ms
8 ae2-70g.cr1.lhr1.uk.nlayer.net (69.22.139.62) 0.530 ms 0.530 ms 0.545 ms
9 xe-4-0-0.cr1.ams2.nl.nlayer.net (69.22.142.95) 7.584 ms 7.601 ms 7.574 ms
10 as23352.xe-4-2-0-105.cr1.ams2.nl.nlayer.net (69.22.139.121) 7.444 ms
xe-1-3-0-111.cr1.ams2.nl.nlayer.net (69.22.139.2) 7.139 ms 7.094 ms
11 52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 8.454 ms
51.vlan.ar1.ams3.nl.scnet.net (205.234.220.194) 15.485 ms
52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 7.958 ms
12 cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.585 ms 7.445 ms
ge0-3.aggrb3.ams3.nl.scnet.net (205.234.220.229) 15.803 ms
13 cf-199-27-134-74.cloudflare.com (199.27.134.74) 7.073 ms 7.145 ms 7.089 ms

Re: VM Routing and CloudFlare.com
 

https://www.cableforum.co.uk/images/local/2011/04/2.png

Goes via NL for me

Re: VM Routing and CloudFlare.com
 

What package are you on?
I am on the 50Mb with modem

I just did a fresh trace route, which is below.

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.

C:\Users\Owner>tracert dev.social-mates.co.uk

Tracing route to dev.social-mates.co.uk [199.27.135.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 5 ms 5 ms 5 ms 10.159.144.1
3 8 ms 7 ms 7 ms nott-core-1a-ge-310-2385.network.virginmedia.net
[82.13.49.221]
4 10 ms 9 ms 9 ms leed-bb-1a-as0-0.network.virginmedia.net [213.10
5.174.169]
5 15 ms 15 ms 14 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 15 ms 15 ms 15 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.10
5.159.6]
7 14 ms 16 ms 15 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.
184.6]
8 87 ms 87 ms 87 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.11
5.61]
9 88 ms 87 ms 87 ms ae2-30g.ar2.iad1.us.nlayer.net [69.31.31.186]
10 88 ms 87 ms 89 ms as13335.ge-0-0-33.ar2.iad1.us.nlayer.net [69.31.
30.26]
11 87 ms 110 ms 87 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.

C:\Users\Owner>

Re: VM Routing and CloudFlare.com
 

goes via US for me

Re: VM Routing and CloudFlare.com
 

Goes via NL for me.

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Administrator>tracert 199.27.135.74

Tracing route to cf-199-27-135-74.cloudflare.com [199.27.135.74]
over a maximum of 30 hops:

1 59 ms 57 ms 80 ms cpc8-bary4-2-0-gw.5-1.cable.virginmedia.com [82.
8.40.1]
2 54 ms 11 ms 57 ms cdif-core-1a-ge-300-935.network.virginmedia.net
[62.254.252.29]
3 108 ms 40 ms 213 ms brhm-bb-1a-as0-0.network.virginmedia.net [212.43
.163.109]
4 12 ms 57 ms 57 ms manc-bb-1b-ae5-0.network.virginmedia.net [213.10
5.174.186]
5 17 ms 19 ms 61 ms popl-bb-1a-as4-0.network.virginmedia.net [212.43
.162.85]
6 62 ms 21 ms 17 ms popl-tmr-1-ae4-0.network.virginmedia.net [213.10
5.159.2]
7 72 ms 69 ms 69 ms amst-ic-1-as0-0.network.virginmedia.net [213.105
.175.6]
8 72 ms 69 ms 69 ms ams-ix.ae1.cr1.ams2.nl.nlayer.net [195.69.145.21
9]
9 69 ms 69 ms 69 ms ae3-60g.cr1.ams2.nl.nlayer.net [69.22.139.238]
10 69 ms 69 ms 69 ms xe-1-3-0-111.cr1.ams2.nl.nlayer.net [69.22.139.2
]
11 27 ms 73 ms 73 ms 51.vlan.ar1.ams3.nl.scnet.net [205.234.220.194]

12 27 ms 45 ms 73 ms ge0-4.aggrB3.ams3.nl.scnet.net [205.234.220.231]

13 30 ms 73 ms 27 ms cf-199-27-131-10.cloudflare.com [199.27.131.10]

14 73 ms 27 ms 73 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.

C:\Documents and Settings\Administrator>

Re: VM Routing and CloudFlare.com
 

I'm on 50 with a superhub

Re: VM Routing and CloudFlare.com
 

im on 50 with modem

Re: VM Routing and CloudFlare.com
 

maybe the modems make it route via the U.S. for some reason?

Re: VM Routing and CloudFlare.com
 

wont make a difference, the routing is done by VMs equipment, look at the hops, these re-direct the packets, mine has a different route to yours, and some of VMs equipment is directing the traffic to the US, and some to NL

Re: VM Routing and CloudFlare.com
 

Could this be down to using non-VM DNS servers?

I know some sites will punt you onto their american load balancers if you are using the google dns servers.

Re: VM Routing and CloudFlare.com
 

I had to check yours twice, very dodgy latency VM side made me think was .us ;P

but yeah your result another example of how routing is luck of the draw on an isp as different ip ranges get routed differently.

on the .us fubared routes VM appear to have a direct link with nlayer who do have a EU presence so it seems quite bizarre, or maybe the arrangement is only a .us handover. For a large isp VM seem to have poor peering.

Mine below.

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 7 ms 8 ms 11 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 9 ms 5 ms 7 ms leic-core-1a-ae3-3495.network.virginmedia.net [8
2.3.33.101]
4 14 ms 8 ms 12 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 15 ms 17 ms 18 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 16 ms 14 ms 17 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.10
5.159.6]
7 18 ms 17 ms 16 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.
184.6]
8 88 ms 87 ms 112 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.11
5.61]
9 85 ms 97 ms 88 ms ae2-30g.ar2.iad1.us.nlayer.net [69.31.31.186]
10 87 ms 90 ms 90 ms as13335.ge-0-0-33.ar2.iad1.us.nlayer.net [69.31.
30.26]
11 86 ms 88 ms 86 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]

---------- Post added at 22:02 ---------- Previous post was at 21:59 ----------

I suggest getting back to cloudflare and ask them what this EU route is, then you can go back to VM and say I know this route exists why arent you using it. Ask VM's NOC or tier2 forum support I wouldnt bother on the phones.

cloudflare probably could fix it themselves but it sounds like they not motivated enough.

---------- Post added at 22:06 ---------- Previous post was at 22:02 ----------

I tried tracerouting to 2 ip's found on pip's trace. It seems that direct ams-ix link is simply disabled for a chunk of VM's network this did go without .us but had to go over above.net 3rd party transit. So this shows VM have a alternate EU route available via abovenet.

Tracing route to ams-ix-2.microsoft.com [195.69.145.21]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 6 ms 7 ms 6 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 9 ms 6 ms 7 ms leic-core-1a-ae3-2232.network.virginmedia.net [8
2.3.33.49]
4 15 ms 11 ms 11 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 14 ms 15 ms 15 ms 213.152.245.53
6 15 ms 47 ms 15 ms ge-4-2-0.mpr1.lhr3.uk.above.net [64.125.27.157]

7 23 ms 25 ms 22 ms so-0-1-0.mpr1.ams1.nl.above.net [64.125.27.221]

8 21 ms 22 ms 23 ms xe-0-0-0.er1.ams1.nl.above.net [64.125.26.81]
9 25 ms 32 ms 23 ms ams-ix-2.microsoft.com [195.69.145.21]

and

Tracing route to amst-ic-1-as0-0.network.virginmedia.net [213.105.175.6]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 26 ms 7 ms 5 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 21 ms 18 ms 14 ms leic-core-1a-ae3-2232.network.virginmedia.net [8
2.3.33.49]
4 43 ms 12 ms 10 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 15 ms 14 ms 17 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 15 ms 15 ms 10 ms popl-bb-1a-ae0-0.network.virginmedia.net [213.10
5.174.229]
7 16 ms 13 ms 12 ms popl-tmr-1-ae4-0.network.virginmedia.net [213.10
5.159.2]
8 23 ms 21 ms 20 ms amst-ic-1-as0-0.network.virginmedia.net [213.105
.175.6]

personally I think its all down to saturation of links or possibly down to trying to have active redundancy, I still get routed via leeds to everywhere which suggests the internal VM network has a saturation point forcing that reroute, and after that then there is all this alternate routing going on which suggests VM do not have the capacity to route their entire customer base on optimal routing.

Re: VM Routing and CloudFlare.com
 

A basic understanding of BGP will fill in most of the gaps here.

EDIT: As will this: http://www.cisco.com/en/US/tech/tk36...80094431.shtml

Is this service affecting? Doesn't look like an incorrect route this seems to be an anycasted network.

---------- Post added at 22:19 ---------- Previous post was at 22:13 ----------

Shoving 4 million subscribers manually through the same route is a waste of capacity.

Shoving 4 million subscribers manually through the same route is a waste of capacity.

You call it 'optimal routing' - see my document above for why, according to the routing protocol, the US route is optimal.

Re: VM Routing and CloudFlare.com
 

I am using OpenDNS.
I have emailed my brother who leaves down the road for a trace route

Just tried through VM DNS and it the same route.

--

I do believe it using anycast.

Re: VM Routing and CloudFlare.com
 

I respect the need to balance across multiple links but of course one would expect an isp to have enough excess capacity cover a route going down in an outage so the fact VM are routing people over america doesnt sit well with me I'm afraid. Using multiple routes is fine, routing people over america to holland is not. I have only had this kind of routing on an isp before once, guess who the isp was?

.
.
.

ntl

nildram, entanet, aaisp, xilo, easynet, freeserve all manage to avoid it.

Re: VM Routing and CloudFlare.com
 

Here's the same tracert via my BT connection

C:\Documents and Settings\Administrator>tracert 199.27.135.74

Tracing route to cf-199-27-135-74.cloudflare.com [199.27.135.74]
over a maximum of 30 hops:

1 53 ms 53 ms 53 ms router.pips [192.168.2.1]
2 59 ms 59 ms 59 ms 217.32.141.35
3 6 ms 59 ms 59 ms 217.32.141.62
4 62 ms 62 ms 62 ms 213.120.161.50
5 61 ms 62 ms 62 ms 217.32.26.54
6 62 ms 62 ms 8 ms 217.32.26.182
7 62 ms 62 ms 8 ms acc2-10GigE-0-0-0-4.bm.21cn-ipp.bt.net [109.159
248.194]
8 68 ms 68 ms 15 ms core2-te0-2-5-0.ealing.ukcore.bt.net [109.159.2
8.130]
9 65 ms 12 ms 65 ms transit1-xe0-1-0.ealing.ukcore.bt.net [194.72.9
234]
10 65 ms 66 ms 66 ms t2c1-ge14-0-0.uk-eal.eu.bt.net [166.49.168.25]
11 66 ms 66 ms 66 ms t2c2-p3-0-1.uk-lon1.eu.bt.net [166.49.208.110]
12 66 ms 66 ms 66 ms t2a1-ge7-0.uk-lon1.eu.bt.net [166.49.135.49]
13 66 ms 69 ms 67 ms xe-0-3-0.cr1.lhr1.uk.nlayer.net [195.66.224.37]

14 66 ms 13 ms 66 ms ae2-70g.cr1.lhr1.uk.nlayer.net [69.22.139.62]
15 73 ms 72 ms 72 ms xe-4-0-0.cr1.ams2.nl.nlayer.net [69.22.142.95]
16 76 ms 74 ms 73 ms as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net [69
22.139.119]
17 73 ms 72 ms 74 ms ge0-3.aggrB3.ams3.nl.scnet.net [205.234.220.229

18 20 ms 20 ms 73 ms cf-199-27-131-10.cloudflare.com [199.27.131.10]

19 72 ms 19 ms 72 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.

Re: VM Routing and CloudFlare.com
 

It's how routing works - the more complex your links to the outside world the higher the chance of 'sub-optimal' routes being optimal according to the EGP.

ntl's network and in turn VM's have a ton of paths out of the network, loads of full table transits and multiple peering points.

Re: VM Routing and CloudFlare.com
 

ntl have a history of rubbish like this, I remember been routed to france then germany to traffic to the usa years back. Thanks for the attempt at educating me on BGP but BGP as shown in the document you linked ultimately works how its been configured to work. So VM have made a concious decision to prefer US routing over EU routing to a EU destination.

Re: VM Routing and CloudFlare.com
 

I also did a trace from traceroute.org
Use BT and Demon network.

BT being a bigger ISP then Demon I would think.
They both got to NL and I am sure other ISPs do

So where does VM get the idea it is a optimal routing when:
A. Other ISP route to NL, so that a total of 5 different companies I tried and they all route to NL, so that 5 to 1 send to NL.

B. Sending customer(s) to the USA, which takes long 40ms+ to load (I know it not long but loading time counts)

Also I heard hasn't gpt the best of US connections in the evening, it make thing worst?

But some VM customers get NL, I haven't had time to read your link yet Ignitionnet.

So is this VM making a half hearted attempt at saving money

Re: VM Routing and CloudFlare.com
 

Ignition I dont understand you at times, I know you know its wrong because not too long ago you yourself was moaning about VM's peering situation. But then randomly you will come out and fight their corner. It looks clear for whatever reason VM have weighted the US nlayer link a higher priority than both abovenet and ams-ix, as to why we dont know. Likely financial tho.

Re: VM Routing and CloudFlare.com
 

No conscious routing decision made, this is an anycast network, it's not an EU destination it is advertised out of multiple geographical areas.

Do recheck the BGP link I posted, it will make everything make more sense.

In summary according to BGP there's nothing to choose between both paths, so it's entirely down to the internal routing within the VM network to the external paths. The internal VM network will see two equal cost paths to the end destination so will take whatever is its preferred path internally to the transit.

It's quite possible that forcing parts of the network to use the EU node is actually what requires a conscious decision.

Re: VM Routing and CloudFlare.com
 

BGP works how its configured you know that, each route is applied weighting to work to the isp's preference.

Sorry I missed the anycast part, however this begs the question still how this issue is not affecting other isp's. If we remove all the jitter from pip's trace his is clearly closer so should be a favourable destination for anycast.

How do you know both paths are configured with equal priority?

Re: VM Routing and CloudFlare.com
 

Both links use nLayer. Both routes are going to the same IP address 199.27.135.74, neither is being preferred.

The AS Path to that destination is the same length on both, so the routing decision comes down to which route internally within the VM network to get to the transit is the best option, which will vary depending where on the network you are and your proximity to the two links in question.

popl-bb-1a has the following paths to get to nlayer, which is, actually, all the VM network cares about in this case:

popl-tmr-1
amst-ic-1
ams-ix.ae1.cr1.ams2.nl.nlayer.net

popl-bb-1b
popl-tmr-2
tele-ic-2
eqix.xe-3-3-0.cr2.iad1.us.nlayer.net

Each hop will have an IS-IS cost, it chooses the shortest path - Amsterdam.

popl-bb-1b has the following options:

popl-tmr-2
tele-ic-2
eqix.xe-3-3-0.cr2.iad1.us.nlayer.net

popl-bb-1a
popl-tmr-1
amst-ic-1
ams-ix.ae1.cr1.ams2.nl.nlayer.net

Again - per the routing protocol it chooses the shortest path - Equinix.

This is not some manual tuning to avoid costs. I'm sticking up for VM on this because you are wrong with your accusations of manual routing for cost purposes (in this case).

---------- Post added at 23:25 ---------- Previous post was at 23:19 ----------

Anycast doesn't work like that, it doesn't measure latency, decisions are made by BGP and whatever IGPs the ISPs are using. I have no idea at all of the relevance of latency in pip's case, EIGRP is the only routing protocol that comes to mind that uses latency in routing decisions, VM use an IP/IS-IS/MPBGP network. He goes to Amsterdam because his route towards nlayer goes via popl-bb-1a, so the lowest metric route internally is via Amsterdam rather than going via popl-bb-1b to go via Equinix.

Perhaps VM are the only operator who peer with nlayer in multiple places?

Re: VM Routing and CloudFlare.com
 

But I don't have a case!
https://www.cableforum.co.uk/images/...2011/05/90.gif

Re: VM Routing and CloudFlare.com
 

Anyway I hope this clears things up. I'm not sticking up for anyone, I'm pointing out that this anomaly, though unwanted, is everything working as intended.

As a final thought why would VM intentionally route to the same location via settlement free peering at New York rather than settlement free peering at Amsterdam?

Re: VM Routing and CloudFlare.com
 

I was more asking if VM was doing it to save money.

---------- Post added at 23:33 ---------- Previous post was at 23:29 ----------

I have 1 more questions.

They say they are going to bring London, Paris and Frankfurt DC online using the same setup, will that make any differents, or will it be the same?

Re: VM Routing and CloudFlare.com
 

Nope, it's a side-effect of how routing works. VM aren't paying a metered rate for traffic either way.

Excuse the technobabble it's kinda a technical issue.

I guess if it's causing a problem reaching out to the guys via the forums or Twitter might be a good move. They can modify routing manually.

---------- Post added at 23:36 ---------- Previous post was at 23:35 ----------

Might do, unsure how VM's network is constructed to that detail but if they peer with nLayer in Paris and Frankfurt they'll have another 2 routes to get to that IP address, one of which might be preferable to the US datacentre.

Re: VM Routing and CloudFlare.com
 

that explanation is making sense so pip just happens to be nearer to the route that has the nl route so for him gets picked.

This may go some way to explaining why VM's network has more routing oddities as well since on a adsl isp everyone tends to go to one central routing point (usually in london) whilst VM have a national network which can drop you off at different points.

Re: VM Routing and CloudFlare.com
 

Correct.

Most ADSL is centralised as their subscribers all terminate on LNS which live in data centres, Easynet's peering is quite centralised on London for historical reasons.

VM have transits running from all over the place, Manchester, Guildford, Winnersh, Docklands, Brentford, etc, etc. The network shifts hundreds of Gbit/s and to be funnelling all of that through London is neither feasible nor intelligent. Routers running with 4 x 40GigE port channels for the sake of centralising would be phenomenally expensive. Getting traffic off the network as quickly as reasonably possible reduces capacity requirements internally as well as keeping routing more available as more exit points.

All that said VM's execution of this strategy can leave a lot to be desired at times.

Not sticking up for anyone, they just happen to be right this time. Nearly everyone is right now and then, I'll be joining the chorus when they next have a routing mishap due to their own traffic management, no fear ;)

Re: VM Routing and CloudFlare.com
 

Ah, now I understand why different VM customers getting the different route.
I thought it was all centalised.

Thanks for the further info :)

Re: VM Routing and CloudFlare.com
 

You are one.

---------- Post added at 14:09 ---------- Previous post was at 14:08 ----------

No worries!

This shouldn't make much difference performance wise to be honest - does the page load up fairly quickly regardless?

Re: VM Routing and CloudFlare.com
 

It is ok.
It is a bit quicker (not by much thought) to go direct to the hosting provider and not via CloudFlare as the server is in the UK.

I am testing them, to a view to start using them

Re: VM Routing and CloudFlare.com
 

If the twitter team can change routing I suggest trying them as ignition said.

Re: VM Routing and CloudFlare.com
 

I have contacted an admin on the VM forum, I will see how that goes first.
Then if that, doesn't go to plan, I will contact the twitter team.

Re: VM Routing and CloudFlare.com
 

evidence backing up what ignition has said I have a route going to one of my servers going via the amsterdam peering link. So the route is available if picked by BGP.

Re: VM Routing and CloudFlare.com
 

Hi,

Now I am confused :confused:, my brother just sent me a trace route, he lives about 3 minutes by car, about 10/15 minutes walk, and we both live in the LN6 area. He is on 10mb I am on 50mb.

So I thought we would of gone the same way, but we don't he goes to NL I go to US.

This is his trace route

C:\Users\James>tracert dev.social-mates.co.uk

Tracing route to dev.social-mates.co.uk [199.27.134.74]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms cf-199-27-134-74.cloudflare.com [199.27.134.74]
2 7 ms 6 ms 7 ms 10.234.100.1
3 12 ms 11 ms 11 ms nott-cam-2b-v636.network.virginmedia.net [82.13.49.157]
4 11 ms 10 ms 17 ms nott-core-1b-ae3-0.network.virginmedia.net [195.182.175.141]
5 14 ms 13 ms 14 ms nrth-bb-1b-as1-0.network.virginmedia.net [213.105.174.173]
6 15 ms 15 ms 61 ms popl-bb-1a-as3-0.network.virginmedia.net [213.105.172.14]
7 15 ms 18 ms 15 ms popl-tmr-1-ae4-0.network.virginmedia.net [213.105.159.2]
8 22 ms 34 ms 46 ms amst-ic-1-as0-0.network.virginmedia.net [213.105.175.6]
9 25 ms 23 ms 29 ms ams-ix.ae1.cr1.ams2.nl.nlayer.net [195.69.145.219]
10 28 ms 54 ms 25 ms ae3-60g.cr1.ams2.nl.nlayer.net [69.22.139.238]
11 35 ms 28 ms 24 ms as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net [69.22.139.119]
12 26 ms 60 ms 44 ms ge0-3.aggrB3.ams3.nl.scnet.net [205.234.220.229]
13 24 ms 26 ms 25 ms cf-199-27-131-10.cloudflare.com [199.27.131.10]
14 26 ms 26 ms 41 ms cf-199-27-134-74.cloudflare.com [199.27.134.74]


Trace complete.

C:\Users\James>

Could this be down to him using a different DOCSIS to me, which may use different equipment?

Re: VM Routing and CloudFlare.com
 

yep hes on DOCSIS1, your on DOCSIS3 which are separate networks

Re: VM Routing and CloudFlare.com
 

wheres your comparison?

its entirely possible to have different routing eg. leics has 2 core routers but the one I am not on has totally different internal VM routing it doesnt go via leeds. (in my opinion superior).

Re: VM Routing and CloudFlare.com
 

I was just seing what route he took and as, we are so close together, I thought we would take the same route. But I forgot he was on a different DOCSIS.