Virgin Akamai Cache Congestion
 

Hello all

I've been looking at a really annoying streaming video quality issue for the past few weeks which i've tracked down to seemingly being an issue with video served from akamai.

For example channel5, seems to hit akadash0.akamaized.net and a traceroute ends at a virgin media address of cur1-melt2-0-0-cust128.12-1.cable.virginm.net (213.106.24.129)

From my investigation it seems to be this cache is congested and causing the streaming quality to drop.

Was hoping someone else more in the know could tell me if i'm onto something here or way off, i don't fancy my chances raising this with CS :D

TIA!

Re: Virgin Akamai Cache Congestion
 

I'll pick this up if you can provide some evidence, and a way to recreate it.

Can anyone else recreate this?

By the way, that reverse DNS lookup isn't correct, though that isn't the cause of your problem. I guess they have reused an old Melton Mowbray IP address but forgot to update the DNS. I can confirm 213.106.24.129 is actually a link to Akamai from VM in Manchester.

Re: Virgin Akamai Cache Congestion
 

Have you tried changing the route by changing the DNS server or trying a VPN

Re: Virgin Akamai Cache Congestion
 

Yeah so if I use cloudflare warp, everything is good.

SP what else can I provide to help? I can reproduce it on my5 and itv hub, I'll try and find out if there's any other on demand that uses akamai and check if I can recreate it there too

Re: Virgin Akamai Cache Congestion
 

What device are you running My5 and ITVHub on? Can you quantify the quality drop? Does it only happen at certain times of day? Also please name some specific assets that show the fault.

BBC iPlayer certainly used to use Akamai, but I don't know if they still do.

Re: Virgin Akamai Cache Congestion
 

Noticed it on both my fire sticks originally, my partner has also mentioned she noticed sky go (i rarely watch anything on it) drops in quality quite often but no way to check anything on the PS4 app.

I've recreated it on my5 chrome on my laptop, using the dev console i can see the segments it's requesting end in "1500" (assume the max bitrate) and that changes down to 800,360,180 before the picture and audio become noticeably worse, the assets then go back to 1500 and the quality improves, then drops etc etc

It does seem to coincide with peak times, it seems to be staying at 1500 right now, i'll test again later this afternoon and try and get some logs / screenshots :)

Re: Virgin Akamai Cache Congestion
 

Frustratingly CH5 don't appear to be sending me to akamai today, which is very strange, i've been hitting https://hwdash.channel5.com/cenc/xxx...PEGDASH_1500_a instead and everything has been fine.

ITV seem to be using https://itvpnpdotcom.cdn1.content.itv.com rather than akamai too

Re: Virgin Akamai Cache Congestion
 

problem solved then :tu:

Re: Virgin Akamai Cache Congestion
 

For now it appears so :D

Spiderpant - just wondering if virgin have any monitoring on these caches, can they see utilisation on the network links to them / the boxes themselves?

I'd imagine they sync their content OOH overnight, but could they somehow end up doing this peak times and causing congestion?

Just trying to get an idea of how these things work heh

Re: Virgin Akamai Cache Congestion
 

The caches are owned and managed by Akamai themselves. VM can only monitor the link capacity, though we'd contact Akamai if we thought the problem lay at their end.

Re: Virgin Akamai Cache Congestion
 

Thanks SP

Interestingly, i'm no longer being routed to the VM cache, and everything is fine...

C:\Users\xx>tracert akadash0.akamaized.net

Tracing route to a1160.w10.akamai.net [88.221.87.131]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms 192.168.88.1
2 12 ms 14 ms 8 ms 10.53.34.249
3 11 ms 12 ms 10 ms gate-core-2a-xe-501-0.network.virginmedia.net [82.2.240.105]
4 * * * Request timed out.
5 * * * Request timed out.
6 27 ms 24 ms 25 ms 86.85-254-62.static.virginmediabusiness.co.uk [62.254.85.86]
7 * * * Request timed out.
8 23 ms 23 ms 26 ms uk-lon03a-ri1-ae-2-0.aorta.net [84.116.135.46]
9 30 ms 24 ms 24 ms 213.46.174.130
10 24 ms 29 ms 26 ms ae8-xcr1.lnt.cw.net [195.2.24.130]
11 30 ms 29 ms 29 ms ae4-ucr1.mcr.cw.net [195.2.27.166]
12 30 ms 30 ms 30 ms 217.161.88.254
13 29 ms 29 ms 27 ms a88-221-87-131.deploy.static.akamaitechnologies.com [88.221.87.131]

Trace complete.

Not really sure what is going on here but pinning down is becoming a nightmare, the trend does seem to be the issue only appears when using the VM cache in particular, very strange.

Re: Virgin Akamai Cache Congestion
 

Maybe somebody already identified the fault and has taken the problem link out of use.

Re: Virgin Akamai Cache Congestion
 

Fingers crossed, i'll keep an eye on it, cheers SP

Re: Virgin Akamai Cache Congestion
 

Are you aware of any issues between BT sport and now tv using apps? Recently at my house (350mbits) I’ve been getting buffering and quality drops. The in-laws have too.

My house:

350mbit (speed test usually good/low latency/jitter)
Shub4
Asus rtac68u
Apple TV 4K (not the recent one) hardwired to router
Nowtv app
Btsport app

In-laws:

100mbit (speed test usually good/low latency/jitter)
Shub3
Asus rtac68u
Samsung 50” connected via 5ghz WiFi - have done Speedtest from tv using fast.com and it can max out the connection.
Nowtv app
Btsport app

Issues started roughly at the same time. I’m area 20, they are burnage/didsbury.

Re: Virgin Akamai Cache Congestion
 

No, I wouldn't know where to begin with that one, sorry

Re: Virgin Akamai Cache Congestion
 

Bit of an update for anyone interested...

I was also having issues with Sky go / now tv / BT sport, HD quality was horrendous and 4K streams buffered and dropped to unwatchable quality.

After a bunch of various troubleshooting i worked out that virgin appear to be doing some kind of DNS hijacking (despite all my kit using 1.1.1.1/8.8.8.8) which was still sending various stuff to other IPs.

I got round this by setting up DoH on my pihole to cloudflare, i'm now getting sent to other IPs for all the above services and the quality seems a lot better.

Still testing but the above appears to have resolved this to an extent

---------- Post added at 10:30 ---------- Previous post was at 10:00 ----------

Just to add a bit of evidence to the above, with cloudflare but wiithout DoH, akadash0.akamaized.net gets me to that internal Virgin host in my OP.

However if i switch to my pihole with DoH enabled, i get sent to a104-77-160-89.deploy.static.akamaitechnologies.com [104.77.160.89]

This is repeatable every time i switch, i'm guessing virgin do this to direct customers to their own Netflix cache etc however it seems to be doing something weird in this case!

Re: Virgin Akamai Cache Congestion
 

So...that sounds exactly like my problem and pihole and cloudflare are also in use.

In-laws:

Router with DNS set to 1.1.1.1 and 1.0.0.1

My house:

PiHole configured with Cloudflare on a docker instance in my Synology NAS

Both had the same issues as you e.g. buffering/incredibly bad 4K (BT ultimate) and really poor picture quality on NowTV sports (with boost) and BT sport HD.

I have set both houses to use VM's DNS for now. Will have to test properly.

Synthetic: Did enabling DoH fix this entirely? Can this be done through the PiHole gui or command line?

Re: Virgin Akamai Cache Congestion
 

It's done via command line, you need to install cloudflared, there's a nice guide for it here - https://docs.pi-hole.net/guides/dns/cloudflared/

As a bit of an update, NOW TV etc is still working fine but BT sport is still poor, i dont believe they're using akamai so it could be another cache with issues but i'm still trying to track that down

---------- Post added at 15:19 ---------- Previous post was at 14:54 ----------

Scratch that, looks like BT use t1-live-euwe2-us-ak-streams-sport.akamaized.net but also seem to use cloudfront, the former appears to hit an ntl host, the latter appears unaffected and resolves to the same IP with DOH and without

Re: Virgin Akamai Cache Congestion
 

Yeah - think i'll stick with VMs DNS until something pops up RE the BT sport issue you're having.

Spiderplant - is this something you could take a peak at?

Re: Virgin Akamai Cache Congestion
 

Sorry, I can't help with this.

Re: Virgin Akamai Cache Congestion
 

Robertus, if you stick a static DNS entry in your pihole for the below hosts, does it resolve the BT quality issue? (might need to reboot your tv etc to flush the DNS cache)

t1-live-euwe2-us-ak-streams-sport.akamaized.net 88.221.87.171

britishtelecom.sc.omtrdc.net 18.202.158.78

Re: Virgin Akamai Cache Congestion
 

Can have a look, though don’t want to faff at the moment with so much sport coming up. Switching to vms own dns seems to have cleared up all of the streaming issues though.

Re: Virgin Akamai Cache Congestion
 

I feel a conspiracy coming on

Re: Virgin Akamai Cache Congestion
 

I wouldn’t say conspiracy, more a fault.

I have tried as you asked Synthetic and bt sport seemed ok for the brief time I tested. Nowtv was ropey again, as you’d expect.

Re: Virgin Akamai Cache Congestion
 

Confirms my theory it's the akamai cache virgin's dns hijacking is sending you to (i think).

No idea how we'd get this raised to anyone in Virgin, i havent tried with virgin's own DNS, it's strange everything is fine with that

I wonder if VM have forgotten they were hijacking other DNS to an old cache, and as most customers will use their default DNS its not been brought up before now

Re: Virgin Akamai Cache Congestion
 

So to confirm:

My in-laws:

Using 1.1.1.1 and 1.0.0.1 as DNS entries on the Asus router

My house:

Using pihole, in docker on my Synology DS918+ and cloudflare (NOT cloudflarED)

Both pretty much started at the same time, roughly 2-3 months ago.

Spiderplant, I realise this isn't your bag, but could provide any assistance to getting this looked at by the relevant team? If not no worries, appreciate your responses regardless.


Strange it worked, wouldn't put it past reverting to type and failling next time I try :lol:

Re: Virgin Akamai Cache Congestion
 

Looks like Virgin are hijacking anything on port 53, which is why using another DNS like 1.1.1.1 / 8.8.8.8 doesnt work, but DoH/DoT does as port 53 is only used from your devices to your local pihole, then encrypted to the upstream DNS with DoH/DoT.

I'm not sure Virgin will even listen to us on this as it appears to be how they redirect customers to their Netflix cache (which works fine) and who knows what other caches, i know they used to do it with youtube aswell but i'm yet to notice an issue there

Re: Virgin Akamai Cache Congestion
 

Synthetic - are you able to post in this thread:

https://community.virginmedia.com/t5.../false#M272628

---------- Post added at 11:21 ---------- Previous post was at 10:38 ----------

Looks like I'll need to figure out how to enable DoH via pihole then.

Assuming this fixes all of my issues. It is strange that this did actually work fine though.

---------- Post added at 11:39 ---------- Previous post was at 11:21 ----------

---------- Post added at 11:39 ---------- Previous post was at 11:39 ----------

is there anyway I can test my DoH - I've installed cloudflared, however using the cloudflare encryption test I get a question mark for DNS sec

Re: Virgin Akamai Cache Congestion
 

Easiest way to test - https://1.1.1.1/help

Yes i'll post my finding over there too :)

Re: Virgin Akamai Cache Congestion
 

Sorry, I wouldn't even know who the relevant team is.

Re: Virgin Akamai Cache Congestion
 

Do you know of any good guides to install cloudflared on a synology NAS running docker and pihole?

Also thanks for posting.

Re: Virgin Akamai Cache Congestion
 

The DNS hijacking is probably also how they enforce blocking of websites for IWF and High Court orders.

Once browsers and OS's begin enabling more secure DNS lookups by default that might get fun.

Re: Virgin Akamai Cache Congestion
 

Just the guides on the pihole wiki, it covers various architectures, as long as you have some kind of cli access you should be able to configure it

https://docs.pi-hole.net/guides/dns/cloudflared/

---------- Post added at 07:01 ---------- Previous post was at 06:51 ----------

Had a quick look,

Looks like Virgin are blocking those at the network level via IP, all of the below addresses take me to a virgin blocked page, slightly worrying as those are some of cloudflare's anycast IPs so they're probably blocking some sites they shouldnt be that are behind cloudflare...

; <<>> DiG 9.16.1-Ubuntu <<>> iptorrents.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16734
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;iptorrents.com. IN A

;; ANSWER SECTION:
iptorrents.com. 81 IN A 104.20.222.73
iptorrents.com. 81 IN A 104.20.223.73
iptorrents.com. 81 IN A 172.67.3.170

Re: Virgin Akamai Cache Congestion
 

So, I have installed Adguard home and pointed my apple tv and PC to it - using DoH etc, BT seems OK however Nowtv is still crap.

Have you seen this behaviour?

https://imgur.com/ZcUj2e1

Re: Virgin Akamai Cache Congestion
 

Not personally, in my testing i found nowtv uses

linear014-gb-hss1-prd-ak.cdn.skycdp.com
and
linear019-gb-hss1-prd-ak.cdn.skycdp.com

for live streaming.

If you do a ping / tracert to those, what IPs do they come back to?

Re: Virgin Akamai Cache Congestion
 

C:\WINDOWS\system32>tracert linear014-gb-hss1-prd-ak.cdn.skycdp.com

Tracing route to a1792.w60.akamai.net [176.255.191.144]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms router.asus.com [18.19.1.1]
2 10 ms 8 ms 9 ms 10.53.39.129
3 9 ms 9 ms 10 ms nott-core-2a-xe-1013-0.network.virginmedia.net [62.255.214.129]
4 * * * Request timed out.
5 * * * Request timed out.
6 13 ms 14 ms 17 ms tcma5-ic-1-ae0-0.network.virginmedia.net [62.253.174.181]
7 13 ms 12 ms 13 ms cur1-runc2-0-0-cust2.1-3.cable.virginm.net [213.105.20.3]
8 * * * Request timed out.
9 11 ms 13 ms 10 ms host16-rangeF-mcdn.cdn.enlba.isp.sky.com [176.255.191.144]





C:\WINDOWS\system32>tracert linear019-gb-hss1-prd-ak.cdn.skycdp.com

Tracing route to a1931.w60.akamai.net [176.255.191.139]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms router.asus.com [18.19.1.1]
2 34 ms 23 ms 18 ms 10.53.39.129
3 45 ms 41 ms 33 ms nott-core-2a-xe-013-0.network.virginmedia.net [62.254.0.153]
4 * * * Request timed out.
5 * * * Request timed out.
6 100 ms 215 ms 184 ms tcma5-ic-1-ae0-0.network.virginmedia.net [62.253.174.181]
7 29 ms 41 ms 72 ms cur1-runc2-0-0-cust2.1-3.cable.virginm.net [213.105.20.3]
8 * * * Request timed out.
9 13 ms 12 ms 10 ms host11-rangeF-mcdn.cdn.enlba.isp.sky.com [176.255.191.139]

Re: Virgin Akamai Cache Congestion
 

So i get a totally different IP Range, and take a much longer route...

Tracing route to a1792.w60.akamai.net [88.221.210.11]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms 192.168.88.1
2 12 ms 13 ms 10 ms 10.53.34.249
3 15 ms 11 ms 9 ms gate-core-2a-xe-501-0.network.virginmedia.net [82.2.240.105]
4 * * * Request timed out.
5 * * * Request timed out.
6 24 ms 23 ms 22 ms 86.85-254-62.static.virginmediabusiness.co.uk [62.254.85.86]
7 * * * Request timed out.
8 25 ms 26 ms 25 ms uk-lon01c-ri2-ae-4-0.aorta.net [84.116.135.50]
9 23 ms 28 ms 21 ms ae-8.a02.londen12.uk.bb.gin.ntt.net [129.250.66.101]
10 26 ms 26 ms 25 ms ae-0.r20.londen12.uk.bb.gin.ntt.net [129.250.3.212]
11 30 ms 26 ms 29 ms ae-24.r00.mancen01.uk.bb.gin.ntt.net [129.250.4.190]
12 25 ms 24 ms 22 ms 212.119.4.98
13 44 ms 33 ms 25 ms r2.man-tcw.exa.net.uk [82.219.45.239]
14 26 ms 22 ms 22 ms r1.man-tcw.exa.net.uk [82.219.44.1]
15 24 ms 27 ms 31 ms r1.lon-the.exa.net.uk [82.219.45.29]
16 26 ms 25 ms 27 ms r2.lon-the.exa.net.uk [82.219.45.94]
17 55 ms 28 ms 30 ms r1.leeds-sc.exa.net.uk [82.219.2.46]
18 46 ms 52 ms 52 ms leeds-transit1.akamai.ce.exa.net.uk [82.219.2.93]
19 31 ms 31 ms 29 ms a88-221-210-11.deploy.static.akamaitechnologies.com [88.221.210.11]

Re: Virgin Akamai Cache Congestion
 

Well that is google dns and my kick ass Asus router for you :p

Re: Virgin Akamai Cache Congestion
 

I get yet another route

Tracing route to a1792.w60.akamai.net [2.22.22.210]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms USGGateway [192.168.1.1]
2 10 ms 10 ms 7 ms 10.53.39.21
3 13 ms 10 ms 10 ms pres-core-2b-xe-0011-0.network.virginmedia.net [62.255.65.125]
4 * * * Request timed out.
5 21 ms 21 ms 18 ms 86.85-254-62.static.virginmediabusiness.co.uk [62.254.85.86]
6 * * * Request timed out.
7 31 ms 19 ms 22 ms be2871.ccr42.lon13.atlas.cogentco.com [154.54.58.185]
8 58 ms 27 ms 27 ms be12489.ccr42.par01.atlas.cogentco.com [154.54.57.70]
9 36 ms 26 ms 24 ms be2103.ccr32.par04.atlas.cogentco.com [154.54.61.22]
10 29 ms 26 ms 25 ms be2151.agr21.par04.atlas.cogentco.com [154.54.61.34]
11 27 ms 27 ms 29 ms akamai.demarc.cogentco.com [149.11.114.234]
12 27 ms 29 ms 26 ms a2-22-22-210.deploy.static.akamaitechnologies.com [2.22.22.210]

Trace complete.

Re: Virgin Akamai Cache Congestion
 

Interesting, I'm sure that's the IP / route I was seeing when I first had issues..

Re: Virgin Akamai Cache Congestion
 

Interestingly enough I've been having issues with the BBC iPlayer constantly buffering every 30-60 seconds for a couple of weeks now.

I've enabled DoT on pfSense which certainly seems to stop me hitting Virgins Akamai caches, I'll have to watch something on the iPlayer tonight and see how things are.

Re: Virgin Akamai Cache Congestion
 

It's cropped up on reddit, enabling DoH / DoT has sorted it there as well.

Re: Virgin Akamai Cache Congestion
 

Is that using 1.1.1.1 DNS? I always found it had poor routing to CDNs including Akamai, compared to Google DNS.

Re: Virgin Akamai Cache Congestion
 

Nah i'm now running my own local unbound server and resolving stuff myself, although i'd expect cloudflare / google to resolve hostnames to the same IP

Re: Virgin Akamai Cache Congestion
 

This is what I get using Google's DNS

https://www.cableforum.uk/board/atta...3&d=1623937394

Pretty slick.

Attachment 29103

Re: Virgin Akamai Cache Congestion
 

Yup, I can confirm the iPlayer now works fine with DoT enabled. No issues streaming anything.

Re: Virgin Akamai Cache Congestion
 

My Google DNS result is virtually the same as above, but 1.1.1.1 takes a longer route via Paris:

tracert a1792.w60.akamai.net

Tracing route to a1792.w60.akamai.net [2.22.22.210]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms RT-AC86U-7358 [192.168.1.1]
2 10 ms 11 ms 11 ms 10.53.39.9
3 15 ms 18 ms 11 ms pres-core-2b-xe-714-0.network.virginmedia.net [80.0.160.153]
4 * * * Request timed out.
5 28 ms 97 ms 108 ms 86.85-254-62.static.virginmediabusiness.co.uk [62.254.85.86]
6 * * * Request timed out.
7 23 ms 32 ms 25 ms be2868.ccr41.lon13.atlas.cogentco.com [154.54.57.153]
8 113 ms 33 ms 29 ms be12497.ccr41.par01.atlas.cogentco.com [154.54.56.130]
9 26 ms 30 ms 28 ms be3183.ccr31.par04.atlas.cogentco.com [154.54.38.66]
10 27 ms 28 ms 25 ms be3169.agr21.par04.atlas.cogentco.com [154.54.37.238]
11 29 ms 30 ms 25 ms akamai.demarc.cogentco.com [149.11.114.234]
12 30 ms 33 ms 36 ms a2-22-22-210.deploy.static.akamaitechnologies.com [2.22.22.210]

I can see the attraction of Cloudflare DNS for those who don't use VM DNS and dislike Google but some of the routing could be better.

Re: Virgin Akamai Cache Congestion
 

So I’m still using VMs DNS. If I set my router to my NAS (which is synology/docker/adguard now) my internet dies. Wondering if it was due to running pihole in docker (have since deleted the container) and a port conflict, however the instance is set to use my nas ip.

Re: Virgin Akamai Cache Congestion
 

why don't you tell your router to use Google dns? (or cloudfare/whatever your preference is)

Re: Virgin Akamai Cache Congestion
 

Because as per synthetics evidence, changing dns provider kills quality on things like Nowtv/btsport. I haven’t tried google dns but I think synthetic had.

Re: Virgin Akamai Cache Congestion
 

Yeah it doesnt matter which you use if you dont have DOH/DoT

Re: Virgin Akamai Cache Congestion
 

Synthetic - have you managed to fully resolve the issue using DoT or DoH for both BT and Now?

Re: Virgin Akamai Cache Congestion
 

Now is working fine but BT I can only resolve by adding static DNS, I'm not sure if my unbound is sending me to a dodgy cache or what

Re: Virgin Akamai Cache Congestion
 

So when I had adguard running, I set my windows pc to my DNS server - however I still experienced issues with now and bt. Bit frustrating really. I’ll try again when I can get my router/dns server playing nicely.

Re: Virgin Akamai Cache Congestion
 

I can't see how "pure" DNS would affect routing as all it does is translate name to IP (and back maybe). If different DNS's resolve differently that may affect things.


You could try the tracepath command which works a little bit differently and doesn't need root privileges on UNIX/LINUX. This can show if packets are fragmented and highlight asymmetric routing though the latter isn't always the problem it used to be.

Re: Virgin Akamai Cache Congestion
 

Whatever the case, using cloudflare kills nowtv/btsport for me.

BT sport ultimate - constant buffering and unwatchable

BT SPORT HD - really bad quality with buffering, just less than ultimate

Nowtv - buffering and really poor quality

As soon as I revert to VM dns it’s fine

Re: Virgin Akamai Cache Congestion
 

Not the best solution but do an nslookup / dig using the virgin servers, then set a static dns in your pihole for that akamai domain so when you're using CF/google, your pihole will send you to the ones you use when you're using the virgin DNS

Or i guess use virgin's DNS if you're not having any trouble with them :)

Re: Virgin Akamai Cache Congestion
 

I've noticed a bit of latency recently, although I've not correlated it to any particular site or route, and I was, unfortunately, dealing with another issue on my MBP that saw seeing my downloads capped to 20Mbit/s, I stumbled across the solution for that by accident though.

I just ran a trace to the akamai server that was mentioned in this thread.

https://www.cableforum.uk/board/atta...3&d=1624725485

MOD EDIT - too large image removed and replaced with smaller image.

Re: Virgin Akamai Cache Congestion
 

So if i use google DNS it sends me to the virgin cache, and everything appear to works as it should now, i wonder if virgin have fixed their mess..

1.1.1.1 still sends me via cogento though

Re: Virgin Akamai Cache Congestion
 

I dont watch any terrestrial tv apart from Wimbledon every year. Yesterday i had the tennis going on my tvs, tablet and phone simultaneously using iplayer and google dns and didnt have any problems.

Re: Virgin Akamai Cache Congestion
 

Going to test this with the 4K stream of the England match on iplayer tonight, using Google DNS

Re: Virgin Akamai Cache Congestion
 

I've just added google#s HTTPS settings, however if I browse to https://1.1.1.1/help it says that i'm not using DoH? It does recognise that I'm using google's service though...

Re: Virgin Akamai Cache Congestion
 

I think that only recognises it's own (cloudflare) DoH

Tried some Sky go / BT sport, using virgins cache and it seems ok, iplayer appears to be getting sent to a cache with an ntl hostname which i'm still having issues with...

If anyone else wants to test the hostnames are vod-progressive.akamaized.net / vod-uhd-uk-live.akamaized.net which resolve to m163-mp1.cvx2-c.lng.dial.ntli.net / m202-mp1.cvx2-c.lng.dial.ntli.net respectively

Re: Virgin Akamai Cache Congestion
 

How did it go?

Re: Virgin Akamai Cache Congestion
 

Perfect, not a single buffer :)

Re: Virgin Akamai Cache Congestion
 

Bit of an update re iplayer..

I may be wrong here, but going off the hostnames it looks like some strange routing on virgin's end, going via new york?

Would explain the huge latency to that server too!

traceroute to 23.206.169.90 (23.206.169.90), 30 hops max, 60 byte packets
1 x.x.x.x1 (x.x.x.x) 0.196 ms 0.120 ms 0.114 ms
2 10.53.34.249 (10.53.34.249) 10.519 ms 13.475 ms 13.712 ms
3 gate-core-2a-xe-7116-0.network.virginmedia.net (80.0.145.13) 13.917 ms 14. 178 ms 14.349 ms
4 * * *
5 * * *
6 86.85-254-62.static.virginmediabusiness.co.uk (62.254.85.86) 29.775 ms 27. 773 ms 30.721 ms
7 ldn-b3-link.ip.twelve99.net (213.248.84.25) 31.562 ms 23.300 ms 24.906 ms
8 ldn-bb1-link.ip.twelve99.net (62.115.120.74) 24.971 ms 22.969 ms ldn-bb4-l ink.ip.twelve99.net (62.115.122.180) 25.864 ms
9 nyk-bb2-link.ip.twelve99.net (62.115.113.20) 99.539 ms 98.796 ms *
10 rest-bb1-link.ip.twelve99.net (62.115.141.244) 132.137 ms 131.152 ms ash-b b2-link.ip.twelve99.net (62.115.112.242) 142.919 ms
11 nyk-bb2-link.ip.twelve99.net (62.115.136.200) 126.897 ms atl-b24-link.ip.tw elve99.net (62.115.125.128) 116.761 ms atl-b24-link.ip.twelve99.net (62.115.125 .191) 113.505 ms
12 ash-bb2-link.ip.twelve99.net (62.115.136.201) 113.644 ms dls-b23-link.ip.tw elve99.net (62.115.123.200) 133.330 ms ash-bb2-link.ip.twelve99.net (62.115.136 .201) 114.849 ms
13 akamai-ic345782-dls-b23.ip.twelve99-cust.net (62.115.176.103) 233.157 ms 2 23.642 ms 219.856 ms
14 dls-b23-link.ip.twelve99.net (80.91.246.75) 147.863 ms ve-uhd-push-uk-live. akamaized.net (23.206.169.90) 132.700 ms 131.578 ms

Does anyone else see the same strange routing / high latency on their VM connection?

--- 23.206.169.90 ping statistics ---
40 packets transmitted, 40 received, 0% packet loss, time 39051ms
rtt min/avg/max/mdev = 128.974/131.397/136.025/1.927 ms

EDIT:

These are examples of the hosts / IPs for the routes that seem to be broken on VM, not sure how we'd get anyone at VM to look into it though!

Live UHD - ve-uhd-push-uk-live.akamaized.net - 2.22.22.112
On demand HD - vod-dash-uk-live.akamaized.net (104.100.95.49)
Live HD - vs-cmaf-push-uk-live.akamaized.net 2.22.22.163

Re: Virgin Akamai Cache Congestion
 

Tracing route to a23-206-169-90.deploy.static.akamaitechnologies.com [23.206.169.90]
over a maximum of 30 hops:

1 4 ms <1 ms <1 ms 18.19.1.1
2 6 ms 10 ms 6 ms 10.53.39.129
3 20 ms 10 ms 13 ms nott-core-2b-xe-014-0.network.virginmedia.net [62.255.214.169]
4 * * * Request timed out.
5 * * * Request timed out.
6 * * * Request timed out.
7 18 ms 15 ms 18 ms m686-mp2.cvx1-b.lis.dial.ntli.net [62.254.42.174]
8 20 ms 18 ms 17 ms 213.46.174.118
9 18 ms 18 ms 17 ms ldn-bb1-link.ip.twelve99.net [62.115.122.188]
10 90 ms 88 ms 95 ms nyk-bb2-link.ip.twelve99.net [62.115.113.20]
11 95 ms * * ash-bb2-link.ip.twelve99.net [62.115.136.201]
12 108 ms 114 ms 107 ms atl-b24-link.ip.twelve99.net [62.115.125.128]
13 125 ms 125 ms 125 ms dls-b23-link.ip.twelve99.net [80.91.246.75]
14 129 ms 220 ms 125 ms akamai-ic345782-dls-b23.ip.twelve99-cust.net [62.115.176.103]
15 126 ms 125 ms 125 ms a23-206-169-90.deploy.static.akamaitechnologies.com [23.206.169.90]

Ping statistics for 23.206.169.90:
Packets: Sent = 40, Received = 40, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 122ms, Maximum = 132ms, Average = 126ms

Re: Virgin Akamai Cache Congestion
 

Cheers General, definitely looks like there's an issue with the routing there

Re: Virgin Akamai Cache Congestion
 

So bit of a follow up / conclusion if anyone is still following / comes across this from google.

tl;dr - akamai won't work properly when using cloudflare DNS with virgin (as of now), use Google's 8.8.8.8 resolver.

The reason for buffering with cloudflare is due to them not supporting ECS, basically this sends your subnet with the DNS packet so the resolver knows to send you to a local / ISP cache.

Google does support this, which is why issues are resolved using google DNS, and i assume virgin's DNS is set to send you to their own local cache anyway.

Re: Virgin Akamai Cache Congestion
 

So to sum up for everyone reading.................buy a proper router and use google's dns and you won't have to put up with any of this shit.

Re: Virgin Akamai Cache Congestion
 

Sort of, if you're not using virgin's DNS, use Google / Quad9 / OpenDNS, just be aware you might run into issues using cloudflare

Re: Virgin Akamai Cache Congestion
 

Finally sorted my issues with Adguard, I could set individual devices to my Synology box - but if I set my WAN DNS entried 1 and 2 so my NAS, I would lose internet connectivity. This same config with pihole worked.

What seems to fix it, is leave the DNS1/2 settings as either blank or normal Google resolver and change the "DNS Server" settings under the Asus LAN tab.

Now, rather than have just my router in the adguard device list, i pretty much have everything :p:

Re: Virgin Akamai Cache Congestion
 

So Synthetic, Cloudflare seems better now - have you tested at all?

Re: Virgin Akamai Cache Congestion
 

Yeah seems ok now, I can only guess virgin upgraded their links to akamai when nit using their caches.

I'm using nextdns without edns myself and I've had no issues :)

Re: Virgin Akamai Cache Congestion
 

Ah, not heart of next dns, is it better?

Re: Virgin Akamai Cache Congestion
 

Me neither. Why not use google dns?

Re: Virgin Akamai Cache Congestion
 

For me it is, better routing / latency than cloudflare / google, they have a web panel where you can manage things like adblocking, block security threats etc

Re: Virgin Akamai Cache Congestion
 

Have given it a try, cheers pal.

Re: Virgin Akamai Cache Congestion
 

Are you still using next dns synthetic?

Re: Virgin Akamai Cache Congestion
 

I'm not, i've recently moved back to using a local pihole with cloudflare as upstream

Re: Virgin Akamai Cache Congestion
 

Any streaming/buffering issues?

Which addresses are you using?

Re: Virgin Akamai Cache Congestion
 

Not that I've noticed nope!

I'm using the usual 1.1.1.1 / 1.0.0.1 addresses.

Are you seeing issues again?