VM Routing and CloudFlare.com
 

Hi,

I signed up to CloudFlare.com a few days ago, and have noticed my trace routes go via the US and not via Amsterdam, NL, like they should.

So I contacted CloudFlare and they say: 'Virgin Media, for some reason, is preferring a US peering route to a European one'.

So I am wondering who I can get in contact with about this?
I have nothing against the off-shore or Customer service, but I think it could be over there heads.

Virgin Media connection
C:\Users\Owner>tracert dev.social-mates.co.uk

Tracing route to dev.social-mates.co.uk [199.27.135.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 58 ms 8 ms 9 ms 10.xxx.144.1
3 7 ms 8 ms 8 ms nott-core-1a-ge-310-2385.network.virginmedia.net [82.13.49.221]
4 10 ms 11 ms 9 ms leed-bb-1a-as0-0.network.virginmedia.net [213.105.174.169]
5 14 ms 17 ms 15 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253.185.238]
6 16 ms 14 ms 15 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.105.159.6]
7 14 ms 15 ms 15 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.184.6]
8 156 ms 88 ms 89 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.115.61]
9 97 ms 102 ms 123 ms as23352.xe-2-1-0-82.cr2.iad1.us.nlayer.net [69.31.30.102]
10 92 ms 89 ms 89 ms unassigned.midatlanticbb.com [66.225.125.1]
11 88 ms 89 ms 89 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]

So I went to traceroute.org and did 3 trace routes from 3 different sites in the UK, which are below.

If you noticed the below 3 all go via: ge0-3.aggrb3.ams3.nl.scnet.net
Which is the Netherlands.

If I do a trace route via http://www.xciv.org/cgi-bin/traceroute
Put in: dev.social-mates.co.uk
Trace route is:

traceroute to dev.social-mates.co.uk (199.27.135.74): 1-30 hops, 38 byte packets
1 ge-0-0-3-113.br1.ix1lon.uk.as34706.net (194.145.196.130) 2.54 ms (ttl=254!) 2.51 ms (ttl=254!) 2.74 ms (ttl=254!)
2 ge-0-0-0-90.br1.thelon.uk.as34706.net (194.145.196.227) 28.7 ms (ttl=253!) 29.3 ms (ttl=253!) 30.1 ms (ttl=253!)
3 po3-1363-cr0.tch.uk.as6908.net (78.41.155.50) 1.37 ms 1.57 ms 1.48 ms
4 te3-4-3501-cr0.nik.nl.as6908.net (78.41.154.46) 7.30 ms 7.44 ms 7.36 ms
5 xe-4-3.r01.amstnl02.nl.bb.gin.ntt.net (81.20.64.41) 8.06 ms 7.91 ms 7.63 ms
6 xe-3-0-1.ar1.ams3.nl.nlayer.net (69.22.139.204) 15.1 ms xe-3-0-3.ar1.ams3.nl.nlayer.net (69.22.139.202) 16.9 ms 15.4 ms
7 as23352.vlan-102.ar1.ams3.nl.nlayer.net (69.22.139.123) 15.2 ms 14.9 ms 14.7 ms
8 ge0-3.aggrB3.ams3.nl.scnet.net (205.234.220.229) 16.9 ms ge0-4.aggrB3.ams3.nl.scnet.net (205.234.220.231) 16.5 ms ge0-3.aggrB3.ams3.nl.scnet.net (205.234.220.229) 15.2 ms
9 cf-199-27-131-10.cloudflare.com (199.27.131.10) 14.4 ms (ttl=56!) 14.1 ms (ttl=56!) 14.2 ms (ttl=56!)
10 cf-199-27-135-74.cloudflare.com (199.27.135.74) 14.2 ms (ttl=56!) 14.4 ms (ttl=56!) 14.5 ms (ttl=56!)


If I do a trace route via: http://ab.newnet.co.uk/cgi-bin/traceroute
Put in: dev.social-mates.co.uk
Trace route is:

traceroute to dev.social-mates.co.uk (199.27.134.74): 1-30 hops, 38 byte packets
1 telh2.newnet.co.uk (212.87.64.1) 0.214 ms (ttl=64!) 0.203 ms (ttl=64!) 0.199 ms (ttl=64!)
2 7609-1.lan2.newnet.co.uk (212.87.79.65) 0.613 ms 0.528 ms 0.326 ms
3 ldn-s2-rou-1041.UK.eurorings.net (134.222.109.85) 0.525 ms (ttl=252!) 0.541 ms (ttl=252!) 0.450 ms (ttl=252!)
4 xe-0-3-0.cr1.lhr1.uk.nlayer.net (195.66.224.37) 1.14 ms 3.92 ms 1.07 ms
5 ae2-70g.cr1.lhr1.uk.nlayer.net (69.22.139.62) 0.648 ms (ttl=250!) 0.800 ms (ttl=250!) 0.700 ms (ttl=250!)
6 xe-4-0-0.cr1.ams2.nl.nlayer.net (69.22.142.95) 6.89 ms (ttl=247!) 7.17 ms (ttl=247!) 7.07 ms (ttl=247!)
7 as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net (69.22.139.119) 7.64 ms (ttl=247!) as23352.xe-4-2-0-105.cr1.ams2.nl.nlayer.net (69.22.139.121) 7.69 ms (ttl=247!) as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net (69.22.139.119) 7.55 ms (ttl=247!)
8 ge0-6.aggrB3.ams3.nl.scnet.net (205.234.220.235) 10.0 ms (ttl=246!) 52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 7.54 ms (ttl=247!) ge0-6.aggrB3.ams3.nl.scnet.net (205.234.220.235) 8.42 ms (ttl=246!)
9 cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.39 ms (ttl=54!) 7.42 ms (ttl=54!) ge0-5.aggrB3.ams3.nl.scnet.net (205.234.220.233) 7.82 ms (ttl=246!)
10 cf-199-27-134-74.cloudflare.com (199.27.134.74) 7.24 ms (ttl=54!) 7.44 ms (ttl=54!) cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.47 ms (ttl=54!)


If I do a trace route via: http://www.erik.co.uk/lg/
Put in: dev.social-mates.co.uk
Trace route is:

traceroute dev.social-mates.co.uk
(please be patient, this script may take a moment before you see anything)

1 46.37.32.129 (46.37.32.129) 1.279 ms 0.527 ms 5.571 ms
2 thn-gw01.cerberus.net.uk (46.37.32.1) 0.177 ms 0.169 ms 0.163 ms
3 ge-0-3-83.lon-th12gw.spn.kcom.com (62.164.215.9) 0.412 ms 0.409 ms 0.465 ms
4 134.222.146.53 (134.222.146.53) 0.477 ms 0.664 ms 0.465 ms
5 ldn-s10-rou-1041.uk.eurorings.net (134.222.146.52) 0.730 ms 0.767 ms 0.712 ms
6 ldn-s2-rou-1003.uk.eurorings.net (134.222.231.29) 0.730 ms 0.769 ms 0.714 ms
7 xe-0-3-0.cr1.lhr1.uk.nlayer.net (195.66.224.37) 1.288 ms 3.096 ms 0.754 ms
8 ae2-70g.cr1.lhr1.uk.nlayer.net (69.22.139.62) 0.530 ms 0.530 ms 0.545 ms
9 xe-4-0-0.cr1.ams2.nl.nlayer.net (69.22.142.95) 7.584 ms 7.601 ms 7.574 ms
10 as23352.xe-4-2-0-105.cr1.ams2.nl.nlayer.net (69.22.139.121) 7.444 ms
xe-1-3-0-111.cr1.ams2.nl.nlayer.net (69.22.139.2) 7.139 ms 7.094 ms
11 52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 8.454 ms
51.vlan.ar1.ams3.nl.scnet.net (205.234.220.194) 15.485 ms
52.vlan.ar2.ams3.nl.scnet.net (205.234.220.198) 7.958 ms
12 cf-199-27-131-10.cloudflare.com (199.27.131.10) 7.585 ms 7.445 ms
ge0-3.aggrb3.ams3.nl.scnet.net (205.234.220.229) 15.803 ms
13 cf-199-27-134-74.cloudflare.com (199.27.134.74) 7.073 ms 7.145 ms 7.089 ms

Re: VM Routing and CloudFlare.com
 

https://www.cableforum.co.uk/images/local/2011/04/2.png

Goes via NL for me

Re: VM Routing and CloudFlare.com
 

What package are you on?
I am on the 50Mb with modem

I just did a fresh trace route, which is below.

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.

C:\Users\Owner>tracert dev.social-mates.co.uk

Tracing route to dev.social-mates.co.uk [199.27.135.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 5 ms 5 ms 5 ms 10.159.144.1
3 8 ms 7 ms 7 ms nott-core-1a-ge-310-2385.network.virginmedia.net
[82.13.49.221]
4 10 ms 9 ms 9 ms leed-bb-1a-as0-0.network.virginmedia.net [213.10
5.174.169]
5 15 ms 15 ms 14 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 15 ms 15 ms 15 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.10
5.159.6]
7 14 ms 16 ms 15 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.
184.6]
8 87 ms 87 ms 87 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.11
5.61]
9 88 ms 87 ms 87 ms ae2-30g.ar2.iad1.us.nlayer.net [69.31.31.186]
10 88 ms 87 ms 89 ms as13335.ge-0-0-33.ar2.iad1.us.nlayer.net [69.31.
30.26]
11 87 ms 110 ms 87 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.

C:\Users\Owner>

Re: VM Routing and CloudFlare.com
 

goes via US for me

Re: VM Routing and CloudFlare.com
 

Goes via NL for me.

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Administrator>tracert 199.27.135.74

Tracing route to cf-199-27-135-74.cloudflare.com [199.27.135.74]
over a maximum of 30 hops:

1 59 ms 57 ms 80 ms cpc8-bary4-2-0-gw.5-1.cable.virginmedia.com [82.
8.40.1]
2 54 ms 11 ms 57 ms cdif-core-1a-ge-300-935.network.virginmedia.net
[62.254.252.29]
3 108 ms 40 ms 213 ms brhm-bb-1a-as0-0.network.virginmedia.net [212.43
.163.109]
4 12 ms 57 ms 57 ms manc-bb-1b-ae5-0.network.virginmedia.net [213.10
5.174.186]
5 17 ms 19 ms 61 ms popl-bb-1a-as4-0.network.virginmedia.net [212.43
.162.85]
6 62 ms 21 ms 17 ms popl-tmr-1-ae4-0.network.virginmedia.net [213.10
5.159.2]
7 72 ms 69 ms 69 ms amst-ic-1-as0-0.network.virginmedia.net [213.105
.175.6]
8 72 ms 69 ms 69 ms ams-ix.ae1.cr1.ams2.nl.nlayer.net [195.69.145.21
9]
9 69 ms 69 ms 69 ms ae3-60g.cr1.ams2.nl.nlayer.net [69.22.139.238]
10 69 ms 69 ms 69 ms xe-1-3-0-111.cr1.ams2.nl.nlayer.net [69.22.139.2
]
11 27 ms 73 ms 73 ms 51.vlan.ar1.ams3.nl.scnet.net [205.234.220.194]

12 27 ms 45 ms 73 ms ge0-4.aggrB3.ams3.nl.scnet.net [205.234.220.231]

13 30 ms 73 ms 27 ms cf-199-27-131-10.cloudflare.com [199.27.131.10]

14 73 ms 27 ms 73 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.

C:\Documents and Settings\Administrator>

Re: VM Routing and CloudFlare.com
 

I'm on 50 with a superhub

Re: VM Routing and CloudFlare.com
 

im on 50 with modem

Re: VM Routing and CloudFlare.com
 

maybe the modems make it route via the U.S. for some reason?

Re: VM Routing and CloudFlare.com
 

wont make a difference, the routing is done by VMs equipment, look at the hops, these re-direct the packets, mine has a different route to yours, and some of VMs equipment is directing the traffic to the US, and some to NL

Re: VM Routing and CloudFlare.com
 

Could this be down to using non-VM DNS servers?

I know some sites will punt you onto their american load balancers if you are using the google dns servers.

Re: VM Routing and CloudFlare.com
 

I had to check yours twice, very dodgy latency VM side made me think was .us ;P

but yeah your result another example of how routing is luck of the draw on an isp as different ip ranges get routed differently.

on the .us fubared routes VM appear to have a direct link with nlayer who do have a EU presence so it seems quite bizarre, or maybe the arrangement is only a .us handover. For a large isp VM seem to have poor peering.

Mine below.

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 7 ms 8 ms 11 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 9 ms 5 ms 7 ms leic-core-1a-ae3-3495.network.virginmedia.net [8
2.3.33.101]
4 14 ms 8 ms 12 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 15 ms 17 ms 18 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 16 ms 14 ms 17 ms popl-tmr-2-ae5-0.network.virginmedia.net [213.10
5.159.6]
7 18 ms 17 ms 16 ms tele-ic-2-as0-0.network.virginmedia.net [62.253.
184.6]
8 88 ms 87 ms 112 ms eqix.xe-3-3-0.cr2.iad1.us.nlayer.net [206.223.11
5.61]
9 85 ms 97 ms 88 ms ae2-30g.ar2.iad1.us.nlayer.net [69.31.31.186]
10 87 ms 90 ms 90 ms as13335.ge-0-0-33.ar2.iad1.us.nlayer.net [69.31.
30.26]
11 86 ms 88 ms 86 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]

---------- Post added at 22:02 ---------- Previous post was at 21:59 ----------

I suggest getting back to cloudflare and ask them what this EU route is, then you can go back to VM and say I know this route exists why arent you using it. Ask VM's NOC or tier2 forum support I wouldnt bother on the phones.

cloudflare probably could fix it themselves but it sounds like they not motivated enough.

---------- Post added at 22:06 ---------- Previous post was at 22:02 ----------

I tried tracerouting to 2 ip's found on pip's trace. It seems that direct ams-ix link is simply disabled for a chunk of VM's network this did go without .us but had to go over above.net 3rd party transit. So this shows VM have a alternate EU route available via abovenet.

Tracing route to ams-ix-2.microsoft.com [195.69.145.21]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 6 ms 7 ms 6 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 9 ms 6 ms 7 ms leic-core-1a-ae3-2232.network.virginmedia.net [8
2.3.33.49]
4 15 ms 11 ms 11 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 14 ms 15 ms 15 ms 213.152.245.53
6 15 ms 47 ms 15 ms ge-4-2-0.mpr1.lhr3.uk.above.net [64.125.27.157]

7 23 ms 25 ms 22 ms so-0-1-0.mpr1.ams1.nl.above.net [64.125.27.221]

8 21 ms 22 ms 23 ms xe-0-0-0.er1.ams1.nl.above.net [64.125.26.81]
9 25 ms 32 ms 23 ms ams-ix-2.microsoft.com [195.69.145.21]

and

Tracing route to amst-ic-1-as0-0.network.virginmedia.net [213.105.175.6]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms home.gateway2 [192.168.1.1]
2 26 ms 7 ms 5 ms cpc14-leic14-2-0-gw.8-1.cable.virginmedia.com [8
2.30.112.1]
3 21 ms 18 ms 14 ms leic-core-1a-ae3-2232.network.virginmedia.net [8
2.3.33.49]
4 43 ms 12 ms 10 ms leed-bb-1a-as8-0.network.virginmedia.net [213.10
5.172.17]
5 15 ms 14 ms 17 ms popl-bb-1b-as1-0.network.virginmedia.net [62.253
.185.238]
6 15 ms 15 ms 10 ms popl-bb-1a-ae0-0.network.virginmedia.net [213.10
5.174.229]
7 16 ms 13 ms 12 ms popl-tmr-1-ae4-0.network.virginmedia.net [213.10
5.159.2]
8 23 ms 21 ms 20 ms amst-ic-1-as0-0.network.virginmedia.net [213.105
.175.6]

personally I think its all down to saturation of links or possibly down to trying to have active redundancy, I still get routed via leeds to everywhere which suggests the internal VM network has a saturation point forcing that reroute, and after that then there is all this alternate routing going on which suggests VM do not have the capacity to route their entire customer base on optimal routing.

Re: VM Routing and CloudFlare.com
 

A basic understanding of BGP will fill in most of the gaps here.

EDIT: As will this: http://www.cisco.com/en/US/tech/tk36...80094431.shtml

Is this service affecting? Doesn't look like an incorrect route this seems to be an anycasted network.

---------- Post added at 22:19 ---------- Previous post was at 22:13 ----------

Shoving 4 million subscribers manually through the same route is a waste of capacity.

Shoving 4 million subscribers manually through the same route is a waste of capacity.

You call it 'optimal routing' - see my document above for why, according to the routing protocol, the US route is optimal.

Re: VM Routing and CloudFlare.com
 

I am using OpenDNS.
I have emailed my brother who leaves down the road for a trace route

Just tried through VM DNS and it the same route.

--

I do believe it using anycast.

Re: VM Routing and CloudFlare.com
 

I respect the need to balance across multiple links but of course one would expect an isp to have enough excess capacity cover a route going down in an outage so the fact VM are routing people over america doesnt sit well with me I'm afraid. Using multiple routes is fine, routing people over america to holland is not. I have only had this kind of routing on an isp before once, guess who the isp was?

.
.
.

ntl

nildram, entanet, aaisp, xilo, easynet, freeserve all manage to avoid it.

Re: VM Routing and CloudFlare.com
 

Here's the same tracert via my BT connection

C:\Documents and Settings\Administrator>tracert 199.27.135.74

Tracing route to cf-199-27-135-74.cloudflare.com [199.27.135.74]
over a maximum of 30 hops:

1 53 ms 53 ms 53 ms router.pips [192.168.2.1]
2 59 ms 59 ms 59 ms 217.32.141.35
3 6 ms 59 ms 59 ms 217.32.141.62
4 62 ms 62 ms 62 ms 213.120.161.50
5 61 ms 62 ms 62 ms 217.32.26.54
6 62 ms 62 ms 8 ms 217.32.26.182
7 62 ms 62 ms 8 ms acc2-10GigE-0-0-0-4.bm.21cn-ipp.bt.net [109.159
248.194]
8 68 ms 68 ms 15 ms core2-te0-2-5-0.ealing.ukcore.bt.net [109.159.2
8.130]
9 65 ms 12 ms 65 ms transit1-xe0-1-0.ealing.ukcore.bt.net [194.72.9
234]
10 65 ms 66 ms 66 ms t2c1-ge14-0-0.uk-eal.eu.bt.net [166.49.168.25]
11 66 ms 66 ms 66 ms t2c2-p3-0-1.uk-lon1.eu.bt.net [166.49.208.110]
12 66 ms 66 ms 66 ms t2a1-ge7-0.uk-lon1.eu.bt.net [166.49.135.49]
13 66 ms 69 ms 67 ms xe-0-3-0.cr1.lhr1.uk.nlayer.net [195.66.224.37]

14 66 ms 13 ms 66 ms ae2-70g.cr1.lhr1.uk.nlayer.net [69.22.139.62]
15 73 ms 72 ms 72 ms xe-4-0-0.cr1.ams2.nl.nlayer.net [69.22.142.95]
16 76 ms 74 ms 73 ms as23352.xe-0-3-0-104.cr1.ams2.nl.nlayer.net [69
22.139.119]
17 73 ms 72 ms 74 ms ge0-3.aggrB3.ams3.nl.scnet.net [205.234.220.229

18 20 ms 20 ms 73 ms cf-199-27-131-10.cloudflare.com [199.27.131.10]

19 72 ms 19 ms 72 ms cf-199-27-135-74.cloudflare.com [199.27.135.74]


Trace complete.